Transformers: improve timeseries support (#23978)
* extract out the field creation parts * extract out the field creation parts * three math modes * better timeseries support * TestData/Graph: load arrow and zoom to data range (#23764) * Docs: Fix building of docs (#23923) * Docs: Fix building of docs * CircleCI: Fixate grafana/docs-base image revision in job for building docs * Docs: enable packages reference docs for 7-beta (#23953) * added packages reference menu item. * removed the draft flag. * Updated docs by running script. * AlertTab: some ui updates (#23971) * updated the alerting tab. * changed so we use a confirm button. * removed uncommeneted import. * Change to secondary buttons Co-Authored-By: Dominik Prokop <dominik.prokop@grafana.com> * trying to fix issue with panel of undefined. * Fix prettier * Update public/app/features/alerting/AlertTab.tsx Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com> * Docs: Query history 7.0 updates (#23955) * Update docs about query history * Update docs/sources/features/explore/index.md Co-Authored-By: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Metrictank: Fix meta inspector consolidator field names (#23838) to match https://github.com/grafana/metrictank/pull/1798 * Chore: Update Grafana version (#23985) * Update Grafana version * Docs: What's new in 7.0 placeholder (#23987) * Docs: What's new in 7.0 placeholder * Updated makefile * Search: minor fixes (#23984) * Search: Use folder id as key when present * Search: Do not render modals if not open * Enterprise: List 7.0 features (#23956) * CircleCI: Fix triggering of jobs for releases (#23999) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix pagination of issues/PR's in changelog generator (#23997) Fix pagination of issues/PR's in changelog generator * Search: Convert time pickers to CSF (#24002) * updated docs for reporting (#23733) * updated docs * peering comments * Added info about what version test mails requires * Tracing: Fix view bounds after trace change (#23994) * Docs: fix image link (#24011) * Update whats new (#24012) * Chore: Put what's new and release notes URLs in package.json (#24006) * Put what's new and release notes URLs in package.json * Upgrade build pipeline tool * Update changelog for v7.0.0-beta1 (#24007) Co-Authored-By: Marcus Efraimsson <marcus.efraimsson@gmail.com> Co-Authored-By: Andrej Ocenas <mr.ocenas@gmail.com> Co-Authored-By: Hugo Häggmark <hugo.haggmark@grafana.com> Co-authored-by: Tobias Skarhed <1438972+tskarhed@users.noreply.github.com> * verify-repo-update: Fix Dockerfile.deb (#24030) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CircleCI: Upgrade build pipeline tool (#24021) * CircleCI: Upgrade build pipeline tool * Devenv: ignore enterprise (#24037) * Add header icon to Add data source page (#24033) * latest.json: Update testing version (#24038) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix login page redirected from password reset (#24032) * Storybook: Rewrite stories to CSF (#23989) * ColorPicker to CSF format * Convert stories to CSF * Do not export ClipboardButton * Update ConfirmButton * Remove unused imports * Fix feedback * changelog enterprise 7.0.0-beta1 (#24039) * CircleCI: Bump grafana/build-container revision (#24043) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Changelog: Updates changelog with more feature details (#24040) * Changelog: Updates changelog with more feature details * spell fix * spell fix * Updates * Readme update * Updates * Select: fixes so component loses focus on selecting value or pressing outside of input. (#24008) * changed the value container to a class component to get it to work with focus (maybe something with context?). * added e2e tests to verify that the select focus is working as it should. * fixed according to feedback. * updated snapshot. * Devenv: add remote renderer to grafana (#24050) * NewPanelEditor: minor UI twekas (#24042) * Forward ref for tabs, use html props * Inspect: add inspect label to drawer title * Add tooltips to sidebar pane tabs, copy changes * Remove unused import * Place tooltips over tabs * Inspector: dont show transformations select if there is only one data frame * Review * Changelog: Add a breaking change (#24051) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CircleCI: Unpin grafana/docs-base (#24054) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Search: close overlay on Esc press (#24003) * Search: Close on Esc * Search: Increase bottom padding for the last item in section * Search: Move closing search to keybindingsSrv * Search: Fix folder view * Search: Do not move folders if already in folder * Docs: Adds deprecation notice to changelog and docs for scripted dashboards (#24060) * Update CHANGELOG.md (#24047) Fix typo Co-authored-by: Daniel Lee <dan.limerick@gmail.com> * Documentation: Alternative Team Sync Wording (#23960) * Alternative wording for team sync docs Signed-off-by: Joe Elliott <number101010@gmail.com> * Update docs/sources/auth/team-sync.md Co-Authored-By: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Fix misspell issues (#23905) * Fix misspell issues See, $ golangci-lint run --timeout 10m --disable-all -E misspell ./... Signed-off-by: Mario Trangoni <mjtrangoni@gmail.com> * Fix codespell issues See, $ codespell -S './.git*' -L 'uint,thru,pres,unknwon,serie,referer,uptodate,durationm' Signed-off-by: Mario Trangoni <mjtrangoni@gmail.com> * ci please? * non-empty commit - ci? * Trigger build Co-authored-by: bergquist <carl.bergquist@gmail.com> Co-authored-by: Kyle Brandt <kyle@grafana.com> * more tests * remove FieldConfig setting * merged binary and reduce * improve tests * update options after values change * Minor refactoring and polish to UI * Minor fixes Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Andersson <marcus.andersson@grafana.com> Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com> Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Dieter Plaetinck <dieter@grafana.com> Co-authored-by: Torkel Ödegaard <torkel@grafana.com> Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com> Co-authored-by: Emil Tullstedt <sakjur@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> Co-authored-by: Jon Gyllenswärd <jon.gyllensward@grafana.com> Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com> Co-authored-by: Hugo Häggmark <hugo.haggmark@grafana.com> Co-authored-by: Tobias Skarhed <1438972+tskarhed@users.noreply.github.com> Co-authored-by: Leonard Gram <leo@xlson.com> Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com> Co-authored-by: Joe Elliott <joe.elliott@grafana.com> Co-authored-by: Mario Trangoni <mario@mariotrangoni.de> Co-authored-by: bergquist <carl.bergquist@gmail.com> Co-authored-by: Kyle Brandt <kyle@grafana.com>
This commit is contained in:
co-authored by
bergquist
Kyle Brandt
Arve Knudsen
Marcus Andersson
Dominik Prokop
Ivana Huckova
Diana Payton
Dieter Plaetinck
Torkel Ödegaard
Alex Khomenko
Emil Tullstedt
Marcus Efraimsson
Jon Gyllenswärd
Andrej Ocenas
Hugo Häggmark
Tobias Skarhed
Leonard Gram
Alexander Zobnin
Richard Hartmann
Daniel Lee
Joe Elliott
Mario Trangoni
parent
5a20782499
commit
53328718e1
@@ -4,19 +4,27 @@ import { FieldType } from '../../types/dataFrame';
|
||||
import { ReducerID } from '../fieldReducer';
|
||||
import { mockTransformationsRegistry } from '../../utils/tests/mockTransformationsRegistry';
|
||||
import { transformDataFrame } from '../transformDataFrame';
|
||||
import { calculateFieldTransformer } from './calculateField';
|
||||
import { calculateFieldTransformer, CalculateFieldMode } from './calculateField';
|
||||
import { DataFrameView } from '../../dataframe';
|
||||
import { BinaryOperationID } from '../../utils';
|
||||
|
||||
const seriesToTestWith = toDataFrame({
|
||||
const seriesA = toDataFrame({
|
||||
fields: [
|
||||
{ name: 'A', type: FieldType.time, values: [1000, 2000] },
|
||||
{ name: 'B', type: FieldType.number, values: [1, 100] },
|
||||
{ name: 'C', type: FieldType.number, values: [2, 200] },
|
||||
{ name: 'TheTime', type: FieldType.time, values: [1000, 2000] },
|
||||
{ name: 'A', type: FieldType.number, values: [1, 100] },
|
||||
],
|
||||
});
|
||||
|
||||
const seriesBC = toDataFrame({
|
||||
fields: [
|
||||
{ name: 'TheTime', type: FieldType.time, values: [1000, 2000] },
|
||||
{ name: 'B', type: FieldType.number, values: [2, 200] },
|
||||
{ name: 'C', type: FieldType.number, values: [3, 300] },
|
||||
{ name: 'D', type: FieldType.string, values: ['first', 'second'] },
|
||||
],
|
||||
});
|
||||
|
||||
describe('calculateField transformer', () => {
|
||||
describe('calculateField transformer w/ timeseries', () => {
|
||||
beforeAll(() => {
|
||||
mockTransformationsRegistry([calculateFieldTransformer]);
|
||||
});
|
||||
@@ -25,28 +33,30 @@ describe('calculateField transformer', () => {
|
||||
const cfg = {
|
||||
id: DataTransformerID.calculateField,
|
||||
options: {
|
||||
// defautls to sum
|
||||
// defaults to `sum` ReduceRow
|
||||
alias: 'The Total',
|
||||
},
|
||||
};
|
||||
|
||||
const filtered = transformDataFrame([cfg], [seriesToTestWith])[0];
|
||||
const filtered = transformDataFrame([cfg], [seriesA, seriesBC])[0];
|
||||
const rows = new DataFrameView(filtered).toArray();
|
||||
expect(rows).toMatchInlineSnapshot(`
|
||||
Array [
|
||||
Object {
|
||||
"A": 1000,
|
||||
"B": 1,
|
||||
"C": 2,
|
||||
"D": "first",
|
||||
"The Total": 3,
|
||||
"A {0}": 1,
|
||||
"B {1}": 2,
|
||||
"C {1}": 3,
|
||||
"D {1}": "first",
|
||||
"The Total": 6,
|
||||
"TheTime": 1000,
|
||||
},
|
||||
Object {
|
||||
"A": 2000,
|
||||
"B": 100,
|
||||
"C": 200,
|
||||
"D": "second",
|
||||
"The Total": 300,
|
||||
"A {0}": 100,
|
||||
"B {1}": 200,
|
||||
"C {1}": 300,
|
||||
"D {1}": "second",
|
||||
"The Total": 600,
|
||||
"TheTime": 2000,
|
||||
},
|
||||
]
|
||||
`);
|
||||
@@ -56,20 +66,25 @@ describe('calculateField transformer', () => {
|
||||
const cfg = {
|
||||
id: DataTransformerID.calculateField,
|
||||
options: {
|
||||
reducer: ReducerID.mean,
|
||||
mode: CalculateFieldMode.ReduceRow,
|
||||
reduce: {
|
||||
reducer: ReducerID.mean,
|
||||
},
|
||||
replaceFields: true,
|
||||
},
|
||||
};
|
||||
|
||||
const filtered = transformDataFrame([cfg], [seriesToTestWith])[0];
|
||||
const filtered = transformDataFrame([cfg], [seriesA, seriesBC])[0];
|
||||
const rows = new DataFrameView(filtered).toArray();
|
||||
expect(rows).toMatchInlineSnapshot(`
|
||||
Array [
|
||||
Object {
|
||||
"Mean": 1.5,
|
||||
"Mean": 2,
|
||||
"TheTime": 1000,
|
||||
},
|
||||
Object {
|
||||
"Mean": 150,
|
||||
"Mean": 200,
|
||||
"TheTime": 2000,
|
||||
},
|
||||
]
|
||||
`);
|
||||
@@ -79,21 +94,86 @@ describe('calculateField transformer', () => {
|
||||
const cfg = {
|
||||
id: DataTransformerID.calculateField,
|
||||
options: {
|
||||
reducer: ReducerID.mean,
|
||||
mode: CalculateFieldMode.ReduceRow,
|
||||
reduce: {
|
||||
include: 'B',
|
||||
reducer: ReducerID.mean,
|
||||
},
|
||||
replaceFields: true,
|
||||
include: 'B',
|
||||
},
|
||||
};
|
||||
|
||||
const filtered = transformDataFrame([cfg], [seriesToTestWith])[0];
|
||||
const filtered = transformDataFrame([cfg], [seriesBC])[0];
|
||||
const rows = new DataFrameView(filtered).toArray();
|
||||
expect(rows).toMatchInlineSnapshot(`
|
||||
Array [
|
||||
Object {
|
||||
"Mean": 1,
|
||||
"Mean": 2,
|
||||
"TheTime": 1000,
|
||||
},
|
||||
Object {
|
||||
"Mean": 100,
|
||||
"Mean": 200,
|
||||
"TheTime": 2000,
|
||||
},
|
||||
]
|
||||
`);
|
||||
});
|
||||
|
||||
it('binary math', () => {
|
||||
const cfg = {
|
||||
id: DataTransformerID.calculateField,
|
||||
options: {
|
||||
mode: CalculateFieldMode.BinaryOperation,
|
||||
binary: {
|
||||
left: 'B',
|
||||
operation: BinaryOperationID.Add,
|
||||
right: 'C',
|
||||
},
|
||||
replaceFields: true,
|
||||
},
|
||||
};
|
||||
|
||||
const filtered = transformDataFrame([cfg], [seriesBC])[0];
|
||||
const rows = new DataFrameView(filtered).toArray();
|
||||
expect(rows).toMatchInlineSnapshot(`
|
||||
Array [
|
||||
Object {
|
||||
"B + C": 5,
|
||||
"TheTime": 1000,
|
||||
},
|
||||
Object {
|
||||
"B + C": 500,
|
||||
"TheTime": 2000,
|
||||
},
|
||||
]
|
||||
`);
|
||||
});
|
||||
|
||||
it('field + static number', () => {
|
||||
const cfg = {
|
||||
id: DataTransformerID.calculateField,
|
||||
options: {
|
||||
mode: CalculateFieldMode.BinaryOperation,
|
||||
binary: {
|
||||
left: 'B',
|
||||
operation: BinaryOperationID.Add,
|
||||
right: '2',
|
||||
},
|
||||
replaceFields: true,
|
||||
},
|
||||
};
|
||||
|
||||
const filtered = transformDataFrame([cfg], [seriesBC])[0];
|
||||
const rows = new DataFrameView(filtered).toArray();
|
||||
expect(rows).toMatchInlineSnapshot(`
|
||||
Array [
|
||||
Object {
|
||||
"B + 2": 4,
|
||||
"TheTime": 1000,
|
||||
},
|
||||
Object {
|
||||
"B + 2": 202,
|
||||
"TheTime": 2000,
|
||||
},
|
||||
]
|
||||
`);
|
||||
|
||||
@@ -4,79 +4,248 @@ import { ReducerID, fieldReducers } from '../fieldReducer';
|
||||
import { getFieldMatcher } from '../matchers';
|
||||
import { FieldMatcherID } from '../matchers/ids';
|
||||
import { RowVector } from '../../vector/RowVector';
|
||||
import { ArrayVector } from '../../vector';
|
||||
import { ArrayVector, BinaryOperationVector, ConstantVector } from '../../vector';
|
||||
import { doStandardCalcs } from '../fieldReducer';
|
||||
import { seriesToColumnsTransformer } from './seriesToColumns';
|
||||
import { getTimeField } from '../../dataframe';
|
||||
import defaults from 'lodash/defaults';
|
||||
import { BinaryOperationID, binaryOperators } from '../../utils/binaryOperators';
|
||||
|
||||
export interface CalculateFieldTransformerOptions {
|
||||
reducer: ReducerID;
|
||||
export enum CalculateFieldMode {
|
||||
ReduceRow = 'reduceRow',
|
||||
BinaryOperation = 'binary',
|
||||
}
|
||||
|
||||
interface ReduceOptions {
|
||||
include?: string; // Assume all fields
|
||||
alias?: string; // The output field name
|
||||
replaceFields?: boolean;
|
||||
reducer: ReducerID;
|
||||
nullValueMode?: NullValueMode;
|
||||
}
|
||||
|
||||
interface BinaryOptions {
|
||||
left: string;
|
||||
operator: BinaryOperationID;
|
||||
right: string;
|
||||
}
|
||||
|
||||
const defaultReduceOptions: ReduceOptions = {
|
||||
reducer: ReducerID.sum,
|
||||
};
|
||||
|
||||
const defaultBinaryOptions: BinaryOptions = {
|
||||
left: '',
|
||||
operator: BinaryOperationID.Add,
|
||||
right: '',
|
||||
};
|
||||
|
||||
export interface CalculateFieldTransformerOptions {
|
||||
// True/False or auto
|
||||
timeSeries?: boolean;
|
||||
mode: CalculateFieldMode; // defaults to 'reduce'
|
||||
|
||||
// Only one should be filled
|
||||
reduce?: ReduceOptions;
|
||||
binary?: BinaryOptions;
|
||||
|
||||
// Remove other fields
|
||||
replaceFields?: boolean;
|
||||
|
||||
// Output field properties
|
||||
alias?: string; // The output field name
|
||||
// TODO: config?: FieldConfig; or maybe field overrides? since the UI exists
|
||||
}
|
||||
|
||||
type ValuesCreator = (data: DataFrame) => Vector;
|
||||
|
||||
export const calculateFieldTransformer: DataTransformerInfo<CalculateFieldTransformerOptions> = {
|
||||
id: DataTransformerID.calculateField,
|
||||
name: 'Add field from calculation',
|
||||
description: 'Use the row values to calculate a new field',
|
||||
defaultOptions: {
|
||||
reducer: ReducerID.sum,
|
||||
mode: CalculateFieldMode.ReduceRow,
|
||||
reduce: {
|
||||
reducer: ReducerID.sum,
|
||||
},
|
||||
},
|
||||
transformer: options => (data: DataFrame[]) => {
|
||||
let matcher = getFieldMatcher({
|
||||
id: FieldMatcherID.numeric,
|
||||
});
|
||||
if (options.include && options.include.length) {
|
||||
matcher = getFieldMatcher({
|
||||
id: FieldMatcherID.byName,
|
||||
options: options.include,
|
||||
});
|
||||
// Assume timeseries should first be joined by time
|
||||
const timeFieldName = findConsistentTimeFieldName(data);
|
||||
|
||||
if (data.length > 1 && timeFieldName && options.timeSeries !== false) {
|
||||
data = seriesToColumnsTransformer.transformer({
|
||||
byField: timeFieldName,
|
||||
})(data);
|
||||
}
|
||||
|
||||
const info = fieldReducers.get(options.reducer);
|
||||
if (!info) {
|
||||
throw new Error(`Unknown reducer: ${options.reducer}`);
|
||||
const mode = options.mode ?? CalculateFieldMode.ReduceRow;
|
||||
let creator: ValuesCreator | undefined = undefined;
|
||||
|
||||
if (mode === CalculateFieldMode.ReduceRow) {
|
||||
creator = getReduceRowCreator(defaults(options.reduce, defaultReduceOptions));
|
||||
} else if (mode === CalculateFieldMode.BinaryOperation) {
|
||||
creator = getBinaryCreator(defaults(options.binary, defaultBinaryOptions));
|
||||
}
|
||||
|
||||
// Nothing configured
|
||||
if (!creator) {
|
||||
return data;
|
||||
}
|
||||
const reducer = info.reduce ?? doStandardCalcs;
|
||||
const ignoreNulls = options.nullValueMode === NullValueMode.Ignore;
|
||||
const nullAsZero = options.nullValueMode === NullValueMode.AsZero;
|
||||
|
||||
return data.map(frame => {
|
||||
// Find the columns that should be examined
|
||||
const columns: Vector[] = [];
|
||||
frame.fields.forEach(field => {
|
||||
if (matcher(field)) {
|
||||
columns.push(field.values);
|
||||
}
|
||||
});
|
||||
|
||||
// Prepare a "fake" field for the row
|
||||
const iter = new RowVector(columns);
|
||||
const row: Field = {
|
||||
name: 'temp',
|
||||
values: iter,
|
||||
type: FieldType.number,
|
||||
config: {},
|
||||
};
|
||||
const vals: number[] = [];
|
||||
for (let i = 0; i < frame.length; i++) {
|
||||
iter.rowIndex = i;
|
||||
row.calcs = undefined; // bust the cache (just in case)
|
||||
const val = reducer(row, ignoreNulls, nullAsZero)[options.reducer];
|
||||
vals.push(val);
|
||||
// delegate field creation to the specific function
|
||||
const values = creator!(frame);
|
||||
if (!values) {
|
||||
return frame;
|
||||
}
|
||||
|
||||
const field = {
|
||||
name: options.alias || info.name,
|
||||
name: getResultFieldNameForCalculateFieldTransformerOptions(options),
|
||||
type: FieldType.number,
|
||||
config: {},
|
||||
values: new ArrayVector(vals),
|
||||
values,
|
||||
};
|
||||
let fields: Field[] = [];
|
||||
|
||||
// Replace all fields with the single field
|
||||
if (options.replaceFields) {
|
||||
const { timeField } = getTimeField(frame);
|
||||
if (timeField && options.timeSeries !== false) {
|
||||
fields = [timeField, field];
|
||||
} else {
|
||||
fields = [field];
|
||||
}
|
||||
} else {
|
||||
fields = [...frame.fields, field];
|
||||
}
|
||||
return {
|
||||
...frame,
|
||||
fields: options.replaceFields ? [field] : [...frame.fields, field],
|
||||
fields,
|
||||
};
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
function getReduceRowCreator(options: ReduceOptions): ValuesCreator {
|
||||
let matcher = getFieldMatcher({
|
||||
id: FieldMatcherID.numeric,
|
||||
});
|
||||
|
||||
if (options.include && options.include.length) {
|
||||
matcher = getFieldMatcher({
|
||||
id: FieldMatcherID.byName,
|
||||
options: options.include,
|
||||
});
|
||||
}
|
||||
|
||||
const info = fieldReducers.get(options.reducer);
|
||||
|
||||
if (!info) {
|
||||
throw new Error(`Unknown reducer: ${options.reducer}`);
|
||||
}
|
||||
|
||||
const reducer = info.reduce ?? doStandardCalcs;
|
||||
const ignoreNulls = options.nullValueMode === NullValueMode.Ignore;
|
||||
const nullAsZero = options.nullValueMode === NullValueMode.AsZero;
|
||||
|
||||
return (frame: DataFrame) => {
|
||||
// Find the columns that should be examined
|
||||
const columns: Vector[] = [];
|
||||
for (const field of frame.fields) {
|
||||
if (matcher(field)) {
|
||||
columns.push(field.values);
|
||||
}
|
||||
}
|
||||
|
||||
// Prepare a "fake" field for the row
|
||||
const iter = new RowVector(columns);
|
||||
const row: Field = {
|
||||
name: 'temp',
|
||||
values: iter,
|
||||
type: FieldType.number,
|
||||
config: {},
|
||||
};
|
||||
const vals: number[] = [];
|
||||
|
||||
for (let i = 0; i < frame.length; i++) {
|
||||
iter.rowIndex = i;
|
||||
row.calcs = undefined; // bust the cache (just in case)
|
||||
const val = reducer(row, ignoreNulls, nullAsZero)[options.reducer];
|
||||
vals.push(val);
|
||||
}
|
||||
|
||||
return new ArrayVector(vals);
|
||||
};
|
||||
}
|
||||
|
||||
function findFieldValuesWithNameOrConstant(frame: DataFrame, name: string): Vector | undefined {
|
||||
if (!name) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
for (const f of frame.fields) {
|
||||
if (f.name === name) {
|
||||
return f.values;
|
||||
}
|
||||
}
|
||||
|
||||
const v = parseFloat(name);
|
||||
if (!isNaN(v)) {
|
||||
return new ConstantVector(v, frame.length);
|
||||
}
|
||||
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function getBinaryCreator(options: BinaryOptions): ValuesCreator {
|
||||
const operator = binaryOperators.getIfExists(options.operator);
|
||||
|
||||
return (frame: DataFrame) => {
|
||||
const left = findFieldValuesWithNameOrConstant(frame, options.left);
|
||||
const right = findFieldValuesWithNameOrConstant(frame, options.right);
|
||||
if (!left || !right || !operator) {
|
||||
return (undefined as unknown) as Vector;
|
||||
}
|
||||
|
||||
return new BinaryOperationVector(left, right, operator.operation);
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find the name for the time field used in all frames (if one exists)
|
||||
*/
|
||||
function findConsistentTimeFieldName(data: DataFrame[]): string | undefined {
|
||||
let name: string | undefined = undefined;
|
||||
for (const frame of data) {
|
||||
const { timeField } = getTimeField(frame);
|
||||
if (!timeField) {
|
||||
return undefined; // Not timeseries
|
||||
}
|
||||
if (!name) {
|
||||
name = timeField.name;
|
||||
} else if (name !== timeField.name) {
|
||||
// Second frame has a different time column?!
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
return name;
|
||||
}
|
||||
|
||||
export function getResultFieldNameForCalculateFieldTransformerOptions(options: CalculateFieldTransformerOptions) {
|
||||
if (options.alias?.length) {
|
||||
return options.alias;
|
||||
}
|
||||
|
||||
if (options.mode === CalculateFieldMode.BinaryOperation) {
|
||||
const { binary } = options;
|
||||
return `${binary?.left ?? ''} ${binary?.operator ?? ''} ${binary?.right ?? ''}`;
|
||||
}
|
||||
|
||||
if (options.mode === CalculateFieldMode.ReduceRow) {
|
||||
const r = fieldReducers.getIfExists(options.reduce?.reducer);
|
||||
if (r) {
|
||||
return r.name;
|
||||
}
|
||||
}
|
||||
|
||||
return 'math';
|
||||
}
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
import { RegistryItem, Registry } from './Registry';
|
||||
|
||||
export enum BinaryOperationID {
|
||||
Add = '+',
|
||||
Subtract = '-',
|
||||
Divide = '/',
|
||||
Multiply = '*',
|
||||
}
|
||||
|
||||
export type BinaryOperation = (left: number, right: number) => number;
|
||||
|
||||
interface BinaryOperatorInfo extends RegistryItem {
|
||||
operation: BinaryOperation;
|
||||
}
|
||||
|
||||
export const binaryOperators = new Registry<BinaryOperatorInfo>(() => {
|
||||
return [
|
||||
{
|
||||
id: BinaryOperationID.Add,
|
||||
name: 'Add',
|
||||
operation: (a: number, b: number) => a + b,
|
||||
},
|
||||
{
|
||||
id: BinaryOperationID.Subtract,
|
||||
name: 'Subtract',
|
||||
operation: (a: number, b: number) => a - b,
|
||||
},
|
||||
{
|
||||
id: BinaryOperationID.Multiply,
|
||||
name: 'Multiply',
|
||||
operation: (a: number, b: number) => a * b,
|
||||
},
|
||||
{
|
||||
id: BinaryOperationID.Divide,
|
||||
name: 'Divide',
|
||||
operation: (a: number, b: number) => a / b,
|
||||
},
|
||||
];
|
||||
});
|
||||
@@ -8,6 +8,7 @@ export * from './labels';
|
||||
export * from './object';
|
||||
export * from './namedColorsPalette';
|
||||
export * from './series';
|
||||
export * from './binaryOperators';
|
||||
export { PanelOptionsEditorBuilder, FieldConfigEditorBuilder } from './OptionsUIBuilders';
|
||||
|
||||
export { getMappedValue } from './valueMappings';
|
||||
|
||||
+5
-2
@@ -1,11 +1,14 @@
|
||||
import { ArrayVector } from './ArrayVector';
|
||||
import { ScaledVector } from './ScaledVector';
|
||||
import { BinaryOperationVector } from './BinaryOperationVector';
|
||||
import { ConstantVector } from './ConstantVector';
|
||||
import { binaryOperators, BinaryOperationID } from '../utils/binaryOperators';
|
||||
|
||||
describe('ScaledVector', () => {
|
||||
it('should support multiply operations', () => {
|
||||
const source = new ArrayVector([1, 2, 3, 4]);
|
||||
const scale = 2.456;
|
||||
const v = new ScaledVector(source, scale);
|
||||
const operation = binaryOperators.get(BinaryOperationID.Multiply).operation;
|
||||
const v = new BinaryOperationVector(source, new ConstantVector(scale, source.length), operation);
|
||||
expect(v.length).toEqual(source.length);
|
||||
// expect(v.push(10)).toEqual(source.length); // not implemented
|
||||
for (let i = 0; i < 10; i++) {
|
||||
@@ -0,0 +1,23 @@
|
||||
import { Vector } from '../types/vector';
|
||||
import { vectorToArray } from './vectorToArray';
|
||||
import { BinaryOperation } from '../utils/binaryOperators';
|
||||
|
||||
export class BinaryOperationVector implements Vector<number> {
|
||||
constructor(private left: Vector<number>, private right: Vector<number>, private operation: BinaryOperation) {}
|
||||
|
||||
get length(): number {
|
||||
return this.left.length;
|
||||
}
|
||||
|
||||
get(index: number): number {
|
||||
return this.operation(this.left.get(index), this.right.get(index));
|
||||
}
|
||||
|
||||
toArray(): number[] {
|
||||
return vectorToArray(this);
|
||||
}
|
||||
|
||||
toJSON(): number[] {
|
||||
return vectorToArray(this);
|
||||
}
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
import { Vector } from '../types/vector';
|
||||
import { vectorToArray } from './vectorToArray';
|
||||
|
||||
export class ScaledVector implements Vector<number> {
|
||||
constructor(private source: Vector<number>, private scale: number) {}
|
||||
|
||||
get length(): number {
|
||||
return this.source.length;
|
||||
}
|
||||
|
||||
get(index: number): number {
|
||||
return this.source.get(index) * this.scale;
|
||||
}
|
||||
|
||||
toArray(): number[] {
|
||||
return vectorToArray(this);
|
||||
}
|
||||
|
||||
toJSON(): number[] {
|
||||
return vectorToArray(this);
|
||||
}
|
||||
}
|
||||
@@ -2,7 +2,7 @@ export * from './AppendedVectors';
|
||||
export * from './ArrayVector';
|
||||
export * from './CircularVector';
|
||||
export * from './ConstantVector';
|
||||
export * from './ScaledVector';
|
||||
export * from './BinaryOperationVector';
|
||||
export * from './SortedVector';
|
||||
|
||||
export { vectorator } from './FunctionalVector';
|
||||
|
||||
+220
-33
@@ -2,19 +2,41 @@ import React, { ChangeEvent } from 'react';
|
||||
import {
|
||||
CalculateFieldTransformerOptions,
|
||||
DataTransformerID,
|
||||
fieldReducers,
|
||||
FieldType,
|
||||
KeyValue,
|
||||
ReducerID,
|
||||
standardTransformers,
|
||||
TransformerRegistyItem,
|
||||
TransformerUIProps,
|
||||
NullValueMode,
|
||||
BinaryOperationID,
|
||||
SelectableValue,
|
||||
binaryOperators,
|
||||
} from '@grafana/data';
|
||||
import { StatsPicker } from '../StatsPicker/StatsPicker';
|
||||
import { Switch } from '../Forms/Legacy/Switch/Switch';
|
||||
import { Input } from '../Input/Input';
|
||||
import { FilterPill } from '../FilterPill/FilterPill';
|
||||
import { HorizontalGroup } from '../Layout/Layout';
|
||||
import {
|
||||
CalculateFieldMode,
|
||||
getResultFieldNameForCalculateFieldTransformerOptions,
|
||||
} from '@grafana/data/src/transformations/transformers/calculateField';
|
||||
import { Select } from '../Select/Select';
|
||||
import defaults from 'lodash/defaults';
|
||||
|
||||
// Copied from @grafana/data ;( not sure how to best support his
|
||||
interface ReduceOptions {
|
||||
include?: string; // Assume all fields
|
||||
reducer: ReducerID;
|
||||
nullValueMode?: NullValueMode;
|
||||
}
|
||||
|
||||
interface BinaryOptions {
|
||||
left: string;
|
||||
operator: BinaryOperationID;
|
||||
right: string;
|
||||
}
|
||||
|
||||
interface CalculateFieldTransformerEditorProps extends TransformerUIProps<CalculateFieldTransformerOptions> {}
|
||||
|
||||
@@ -24,14 +46,20 @@ interface CalculateFieldTransformerEditorState {
|
||||
selected: string[];
|
||||
}
|
||||
|
||||
const calculationModes = [
|
||||
{ value: CalculateFieldMode.BinaryOperation, label: 'Binary operation' },
|
||||
{ value: CalculateFieldMode.ReduceRow, label: 'Reduce row' },
|
||||
];
|
||||
|
||||
export class CalculateFieldTransformerEditor extends React.PureComponent<
|
||||
CalculateFieldTransformerEditorProps,
|
||||
CalculateFieldTransformerEditorState
|
||||
> {
|
||||
constructor(props: CalculateFieldTransformerEditorProps) {
|
||||
super(props);
|
||||
|
||||
this.state = {
|
||||
include: props.options.include || '',
|
||||
include: props.options?.reduce?.include || '',
|
||||
names: [],
|
||||
selected: [],
|
||||
};
|
||||
@@ -41,9 +69,16 @@ export class CalculateFieldTransformerEditor extends React.PureComponent<
|
||||
this.initOptions();
|
||||
}
|
||||
|
||||
componentDidUpdate(oldProps: CalculateFieldTransformerEditorProps) {
|
||||
if (this.props.input !== oldProps.input) {
|
||||
this.initOptions();
|
||||
}
|
||||
}
|
||||
|
||||
private initOptions() {
|
||||
const { input, options } = this.props;
|
||||
const configuredOptions = options.include ? options.include.split('|') : [];
|
||||
const include = options?.reduce?.include || '';
|
||||
const configuredOptions = include.split('|');
|
||||
|
||||
const allNames: string[] = [];
|
||||
const byName: KeyValue<boolean> = {};
|
||||
@@ -78,23 +113,6 @@ export class CalculateFieldTransformerEditor extends React.PureComponent<
|
||||
}
|
||||
}
|
||||
|
||||
onFieldToggle = (fieldName: string) => {
|
||||
const { selected } = this.state;
|
||||
if (selected.indexOf(fieldName) > -1) {
|
||||
this.onChange(selected.filter(s => s !== fieldName));
|
||||
} else {
|
||||
this.onChange([...selected, fieldName]);
|
||||
}
|
||||
};
|
||||
|
||||
onChange = (selected: string[]) => {
|
||||
this.setState({ selected });
|
||||
this.props.onChange({
|
||||
...this.props.options,
|
||||
include: selected.join('|'),
|
||||
});
|
||||
};
|
||||
|
||||
onToggleReplaceFields = () => {
|
||||
const { options } = this.props;
|
||||
this.props.onChange({
|
||||
@@ -103,6 +121,15 @@ export class CalculateFieldTransformerEditor extends React.PureComponent<
|
||||
});
|
||||
};
|
||||
|
||||
onModeChanged = (value: SelectableValue<CalculateFieldMode>) => {
|
||||
const { options, onChange } = this.props;
|
||||
const mode = value.value ?? CalculateFieldMode.BinaryOperation;
|
||||
onChange({
|
||||
...options,
|
||||
mode,
|
||||
});
|
||||
};
|
||||
|
||||
onAliasChanged = (evt: ChangeEvent<HTMLInputElement>) => {
|
||||
const { options } = this.props;
|
||||
this.props.onChange({
|
||||
@@ -111,20 +138,50 @@ export class CalculateFieldTransformerEditor extends React.PureComponent<
|
||||
});
|
||||
};
|
||||
|
||||
onStatsChange = (stats: string[]) => {
|
||||
this.props.onChange({
|
||||
...this.props.options,
|
||||
reducer: stats.length ? (stats[0] as ReducerID) : ReducerID.sum,
|
||||
//---------------------------------------------------------
|
||||
// Reduce by Row
|
||||
//---------------------------------------------------------
|
||||
|
||||
updateReduceOptions = (v: ReduceOptions) => {
|
||||
const { options, onChange } = this.props;
|
||||
onChange({
|
||||
...options,
|
||||
mode: CalculateFieldMode.ReduceRow,
|
||||
reduce: v,
|
||||
});
|
||||
};
|
||||
|
||||
render() {
|
||||
const { options } = this.props;
|
||||
onFieldToggle = (fieldName: string) => {
|
||||
const { selected } = this.state;
|
||||
if (selected.indexOf(fieldName) > -1) {
|
||||
this.onChange(selected.filter(s => s !== fieldName));
|
||||
} else {
|
||||
this.onChange([...selected, fieldName]);
|
||||
}
|
||||
};
|
||||
|
||||
onChange = (selected: string[]) => {
|
||||
this.setState({ selected });
|
||||
const { reduce } = this.props.options;
|
||||
this.updateReduceOptions({
|
||||
...reduce!,
|
||||
include: selected.join('|'),
|
||||
});
|
||||
};
|
||||
|
||||
onStatsChange = (stats: string[]) => {
|
||||
const reducer = stats.length ? (stats[0] as ReducerID) : ReducerID.sum;
|
||||
|
||||
const { reduce } = this.props.options;
|
||||
this.updateReduceOptions({ ...reduce, reducer });
|
||||
};
|
||||
|
||||
renderReduceRow(options?: ReduceOptions) {
|
||||
const { names, selected } = this.state;
|
||||
const reducer = fieldReducers.get(options.reducer);
|
||||
options = defaults(options, { reducer: ReducerID.sum });
|
||||
|
||||
return (
|
||||
<div>
|
||||
<>
|
||||
<div className="gf-form-inline">
|
||||
<div className="gf-form gf-form--grow">
|
||||
<div className="gf-form-label width-8">Field name</div>
|
||||
@@ -145,19 +202,149 @@ export class CalculateFieldTransformerEditor extends React.PureComponent<
|
||||
</div>
|
||||
</div>
|
||||
<div className="gf-form-inline">
|
||||
<div className="gf-form gf-form--grow">
|
||||
<div className="gf-form">
|
||||
<div className="gf-form-label width-8">Calculation</div>
|
||||
<StatsPicker stats={[options.reducer]} onChange={this.onStatsChange} defaultStat={ReducerID.sum} />
|
||||
<StatsPicker
|
||||
allowMultiple={false}
|
||||
className="width-18"
|
||||
stats={[options.reducer]}
|
||||
onChange={this.onStatsChange}
|
||||
defaultStat={ReducerID.sum}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
//---------------------------------------------------------
|
||||
// Binary Operator
|
||||
//---------------------------------------------------------
|
||||
|
||||
updateBinaryOptions = (v: BinaryOptions) => {
|
||||
const { options, onChange } = this.props;
|
||||
onChange({
|
||||
...options,
|
||||
mode: CalculateFieldMode.BinaryOperation,
|
||||
binary: v,
|
||||
});
|
||||
};
|
||||
|
||||
onBinaryLeftChanged = (v: SelectableValue<string>) => {
|
||||
const { binary } = this.props.options;
|
||||
this.updateBinaryOptions({
|
||||
...binary!,
|
||||
left: v.value!,
|
||||
});
|
||||
};
|
||||
|
||||
onBinaryRightChanged = (v: SelectableValue<string>) => {
|
||||
const { binary } = this.props.options;
|
||||
this.updateBinaryOptions({
|
||||
...binary!,
|
||||
right: v.value!,
|
||||
});
|
||||
};
|
||||
|
||||
onBinaryOperationChanged = (v: SelectableValue<string>) => {
|
||||
const { binary } = this.props.options;
|
||||
this.updateBinaryOptions({
|
||||
...binary!,
|
||||
operator: v.value! as BinaryOperationID,
|
||||
});
|
||||
};
|
||||
|
||||
renderBinaryOperation(options?: BinaryOptions) {
|
||||
options = defaults(options, { reducer: ReducerID.sum });
|
||||
|
||||
let foundLeft = !options?.left;
|
||||
let foundRight = !options?.right;
|
||||
const names = this.state.names.map(v => {
|
||||
if (v === options?.left) {
|
||||
foundLeft = true;
|
||||
}
|
||||
if (v === options?.right) {
|
||||
foundRight = true;
|
||||
}
|
||||
return { label: v, value: v };
|
||||
});
|
||||
const leftNames = foundLeft ? names : [...names, { label: options?.left, value: options?.left }];
|
||||
const rightNames = foundRight ? names : [...names, { label: options?.right, value: options?.right }];
|
||||
|
||||
const ops = binaryOperators.list().map(v => {
|
||||
return { label: v.id, value: v.id };
|
||||
});
|
||||
|
||||
return (
|
||||
<div className="gf-form-inline">
|
||||
<div className="gf-form">
|
||||
<div className="gf-form-label width-8">Operation</div>
|
||||
</div>
|
||||
<div className="gf-form">
|
||||
<Select
|
||||
allowCustomValue
|
||||
placeholder="Field or number"
|
||||
options={leftNames}
|
||||
className="min-width-18 gf-form-spacing"
|
||||
value={options?.left}
|
||||
onChange={this.onBinaryLeftChanged}
|
||||
/>
|
||||
<Select
|
||||
className="width-8 gf-form-spacing"
|
||||
options={ops}
|
||||
value={options.operator ?? ops[0].value}
|
||||
onChange={this.onBinaryOperationChanged}
|
||||
/>
|
||||
<Select
|
||||
allowCustomValue
|
||||
placeholder="Field or number"
|
||||
className="min-width-10"
|
||||
options={rightNames}
|
||||
value={options?.right}
|
||||
onChange={this.onBinaryRightChanged}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
//---------------------------------------------------------
|
||||
// Render
|
||||
//---------------------------------------------------------
|
||||
|
||||
render() {
|
||||
const { options } = this.props;
|
||||
|
||||
const mode = options.mode ?? CalculateFieldMode.BinaryOperation;
|
||||
|
||||
return (
|
||||
<div>
|
||||
<div className="gf-form-inline">
|
||||
<div className="gf-form gf-form--grow">
|
||||
<div className="gf-form">
|
||||
<div className="gf-form-label width-8">Mode</div>
|
||||
<Select
|
||||
className="width-18"
|
||||
options={calculationModes}
|
||||
value={calculationModes.find(v => v.value === mode)}
|
||||
onChange={this.onModeChanged}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
{mode === CalculateFieldMode.BinaryOperation && this.renderBinaryOperation(options.binary)}
|
||||
{mode === CalculateFieldMode.ReduceRow && this.renderReduceRow(options.reduce)}
|
||||
<div className="gf-form-inline">
|
||||
<div className="gf-form">
|
||||
<div className="gf-form-label width-8">Alias</div>
|
||||
<Input value={options.alias} placeholder={reducer.name} onChange={this.onAliasChanged} />
|
||||
<Input
|
||||
className="width-18"
|
||||
value={options.alias ?? ''}
|
||||
placeholder={getResultFieldNameForCalculateFieldTransformerOptions(options)}
|
||||
onChange={this.onAliasChanged}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<div className="gf-form-inline">
|
||||
<div className="gf-form gf-form--grow">
|
||||
<div className="gf-form">
|
||||
<Switch
|
||||
label="Replace all fields"
|
||||
labelClass="width-8"
|
||||
|
||||
Reference in New Issue
Block a user