Azure: Basic Logs support (#88025)
* Azure monitor: Basic Logs frontend (#85905) * adds datasource level config for enabling basic logs * add basiclogsquery type to query json * add toggle between basic and analytics * adds basic logs toggle from UI, blocks time picker to only dashboard if basic logs is selected * add check to remove UI if alerting * tests for logsmanagement component * tests for logs query editor * tests for time mangement control * remove unused imports * clears query whenever toggle changes from basic <-> analytics * add test to account for clearning query * Update public/app/plugins/datasource/azuremonitor/components/ConfigEditor/BasicLogsToggle.tsx wording Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * Update public/app/plugins/datasource/azuremonitor/components/LogsQueryEditor/LogsQueryEditor.tsx spelling Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * Update public/app/plugins/datasource/azuremonitor/components/LogsQueryEditor/LogsQueryEditor.tsx spelling Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * update dependency list * clear basic logs if resources change * fix tests --------- Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * Azure Monitor: Basic Logs modal acknowledgement (#86244) * adds datasource level config for enabling basic logs * add basiclogsquery type to query json * add toggle between basic and analytics * adds basic logs toggle from UI, blocks time picker to only dashboard if basic logs is selected * add check to remove UI if alerting * tests for logsmanagement component * tests for logs query editor * tests for time mangement control * remove unused imports * add confirm modal * clears query whenever toggle changes from basic <-> analytics * add test to account for clearning query * adds modal acknowledgement for basic logs query * tests for handling modal logic * basic logs ack type * Update public/app/plugins/datasource/azuremonitor/components/ConfigEditor/BasicLogsToggle.tsx wording Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * Update public/app/plugins/datasource/azuremonitor/components/LogsQueryEditor/LogsQueryEditor.tsx spelling Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * Update public/app/plugins/datasource/azuremonitor/components/LogsQueryEditor/LogsQueryEditor.tsx spelling Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * update dependency list * clear basic logs if resources change * remove modal from config page * remove basic logs query ack type * add modal acknowledgement to toggle between basic and analytics * clear query if resources change * fix tests * fix tests * Update public/app/plugins/datasource/azuremonitor/components/LogsQueryEditor/LogsManagement.tsx Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * fix tests --------- Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * Azure Monitor: Basic Logs Backend (#87653) * fix logic for showingBasicLogsToggle * move to utils function and add basiclogsquery in apply template variable * add backend safeguards for basiclogsqueries * adds support for calling search or query apis based on whether it is basic logs or not * add tests for utils * initial test for basic logs query in the backend * tests for basic logs * remve comment * simplify checks for basic logs * adds fromAlert prop for azure monitor backend services * adds fromAlert check fo basic logs * fix working and empty tags * add telemetry for basic logs * remove import from grafana core package * change fromAlert true in tests * change the way tests catch errors * Update pkg/tsdb/azuremonitor/loganalytics/utils.go Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * Update pkg/tsdb/azuremonitor/loganalytics/utils.go Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * Update pkg/tsdb/azuremonitor/loganalytics/utils.go Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * Update pkg/tsdb/azuremonitor/loganalytics/azure-log-analytics-datasource.go Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * restructure code to only run basic logs checks if basiclogsflag is true * data retention warning * tests for calculate time range * Simplify determining if request is from alerting * Fix lint and bool check * Fix tests * clarify data retention --------- Co-authored-by: Jocelyn <jcolladokuri@microsoft.com> Co-authored-by: jcolladokuri <jocelyncollado52@gmail.com> * Azure Monitor: Basic Logs data volume notification (#88009) * frontend changes for data ingested warning * initial logic for getResource * payload processing * create basicLogs usage function * add utils for converting time and getting the data volume query for basic logs * frontend updates for showing the data ingested for the given query * frontend tests * add check for when no dataIngested is returned * remove backend.logger prints * comment on what function does * fix merge * make resource URI regex case insensitive * add support for workspace variables in basic logs flow * add undefined check * structure and add tests for variable support * Update pkg/tsdb/azuremonitor/loganalytics/azure-log-analytics-datasource.go Co-authored-by: Andreas Christou <andreas.christou@grafana.com> * add tracing for basic logs usage request * clean up data volume query struct * use async/await instead of callback * fix parameters for getApiURL * restrict time on usage query to 8 days max * add time to dependency array to refetch basic logs usage * move time check implementation to backend * fix utils tests --------- Co-authored-by: Jocelyn <jcolladokuri@microsoft.com> Co-authored-by: jcolladokuri <jocelyncollado52@gmail.com> --------- Co-authored-by: jcolladokuri <jcolladokuri@microsoft.com> Co-authored-by: jcolladokuri <jocelyncollado52@gmail.com>
This commit is contained in:
Andreas Christou
co-authored by
Jocelyn
jcolladokuri
Jocelyn
jcolladokuri
parent
0b2fab9967
commit
3eea71cc6b
@@ -21,6 +21,10 @@ import (
|
||||
"github.com/grafana/grafana/pkg/tsdb/azuremonitor/types"
|
||||
)
|
||||
|
||||
func makeQueryPointer(q AzureLogAnalyticsQuery) *AzureLogAnalyticsQuery {
|
||||
return &q
|
||||
}
|
||||
|
||||
func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
fromStart := time.Date(2018, 3, 15, 13, 0, 0, 0, time.UTC).In(time.Local)
|
||||
timeRange := backend.TimeRange{From: fromStart, To: fromStart.Add(34 * time.Minute)}
|
||||
@@ -95,12 +99,14 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
fromAlert bool
|
||||
queryModel backend.DataQuery
|
||||
azureLogAnalyticsQuery AzureLogAnalyticsQuery
|
||||
azureLogAnalyticsQuery *AzureLogAnalyticsQuery
|
||||
Err require.ErrorAssertionFunc
|
||||
}{
|
||||
{
|
||||
name: "Query with macros should be interpolated",
|
||||
name: "Query with macros should be interpolated",
|
||||
fromAlert: false,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
@@ -115,7 +121,7 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
TimeRange: timeRange,
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: AzureLogAnalyticsQuery{
|
||||
azureLogAnalyticsQuery: makeQueryPointer(AzureLogAnalyticsQuery{
|
||||
RefID: "A",
|
||||
ResultFormat: dataquery.ResultFormatTimeSeries,
|
||||
URL: "v1/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/AppInsightsTestDataWorkspace/query",
|
||||
@@ -134,11 +140,12 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
QueryType: dataquery.AzureQueryTypeAzureLogAnalytics,
|
||||
AppInsightsQuery: false,
|
||||
DashboardTime: false,
|
||||
},
|
||||
}),
|
||||
Err: require.NoError,
|
||||
},
|
||||
{
|
||||
name: "Legacy queries with a workspace GUID should use workspace-centric url",
|
||||
name: "Legacy queries with a workspace GUID should use workspace-centric url",
|
||||
fromAlert: false,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
@@ -151,7 +158,7 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
RefID: "A",
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: AzureLogAnalyticsQuery{
|
||||
azureLogAnalyticsQuery: makeQueryPointer(AzureLogAnalyticsQuery{
|
||||
RefID: "A",
|
||||
ResultFormat: dataquery.ResultFormatTimeSeries,
|
||||
URL: "v1/workspaces/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/query",
|
||||
@@ -168,11 +175,12 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
QueryType: dataquery.AzureQueryTypeAzureLogAnalytics,
|
||||
AppInsightsQuery: false,
|
||||
DashboardTime: false,
|
||||
},
|
||||
}),
|
||||
Err: require.NoError,
|
||||
},
|
||||
{
|
||||
name: "Legacy workspace queries with a resource URI (from a template variable) should use resource-centric url",
|
||||
name: "Legacy workspace queries with a resource URI (from a template variable) should use resource-centric url",
|
||||
fromAlert: false,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
@@ -185,7 +193,7 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
RefID: "A",
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: AzureLogAnalyticsQuery{
|
||||
azureLogAnalyticsQuery: makeQueryPointer(AzureLogAnalyticsQuery{
|
||||
RefID: "A",
|
||||
ResultFormat: dataquery.ResultFormatTimeSeries,
|
||||
URL: "v1/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/AppInsightsTestDataWorkspace/query",
|
||||
@@ -202,11 +210,12 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
QueryType: dataquery.AzureQueryTypeAzureLogAnalytics,
|
||||
AppInsightsQuery: false,
|
||||
DashboardTime: false,
|
||||
},
|
||||
}),
|
||||
Err: require.NoError,
|
||||
},
|
||||
{
|
||||
name: "Queries with multiple resources",
|
||||
name: "Queries with multiple resources",
|
||||
fromAlert: false,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
@@ -220,7 +229,7 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
RefID: "A",
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: AzureLogAnalyticsQuery{
|
||||
azureLogAnalyticsQuery: makeQueryPointer(AzureLogAnalyticsQuery{
|
||||
RefID: "A",
|
||||
ResultFormat: dataquery.ResultFormatTimeSeries,
|
||||
URL: "v1/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/AppInsightsTestDataWorkspace/query",
|
||||
@@ -238,11 +247,12 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
QueryType: dataquery.AzureQueryTypeAzureLogAnalytics,
|
||||
AppInsightsQuery: false,
|
||||
DashboardTime: false,
|
||||
},
|
||||
}),
|
||||
Err: require.NoError,
|
||||
},
|
||||
{
|
||||
name: "Query with multiple resources",
|
||||
name: "Query with multiple resources",
|
||||
fromAlert: false,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
@@ -257,7 +267,7 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
TimeRange: timeRange,
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: AzureLogAnalyticsQuery{
|
||||
azureLogAnalyticsQuery: makeQueryPointer(AzureLogAnalyticsQuery{
|
||||
RefID: "A",
|
||||
ResultFormat: dataquery.ResultFormatTimeSeries,
|
||||
URL: "v1/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/AppInsightsTestDataWorkspace/query",
|
||||
@@ -276,11 +286,12 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
QueryType: dataquery.AzureQueryTypeAzureLogAnalytics,
|
||||
AppInsightsQuery: false,
|
||||
DashboardTime: false,
|
||||
},
|
||||
}),
|
||||
Err: require.NoError,
|
||||
},
|
||||
{
|
||||
name: "Query that uses dashboard time",
|
||||
name: "Query that uses dashboard time",
|
||||
fromAlert: false,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
@@ -296,7 +307,7 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
TimeRange: timeRange,
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: AzureLogAnalyticsQuery{
|
||||
azureLogAnalyticsQuery: makeQueryPointer(AzureLogAnalyticsQuery{
|
||||
RefID: "A",
|
||||
ResultFormat: dataquery.ResultFormatTimeSeries,
|
||||
URL: "v1/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/AppInsightsTestDataWorkspace/query",
|
||||
@@ -317,16 +328,127 @@ func TestBuildLogAnalyticsQuery(t *testing.T) {
|
||||
AppInsightsQuery: false,
|
||||
DashboardTime: true,
|
||||
TimeColumn: "TimeGenerated",
|
||||
},
|
||||
}),
|
||||
Err: require.NoError,
|
||||
},
|
||||
{
|
||||
name: "Basic Logs query",
|
||||
fromAlert: false,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
"azureLogAnalytics": {
|
||||
"resources": ["/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/TestDataWorkspace"],
|
||||
"query": "Perf",
|
||||
"resultFormat": "%s",
|
||||
"dashboardTime": true,
|
||||
"timeColumn": "TimeGenerated",
|
||||
"basicLogsQuery": true
|
||||
}
|
||||
}`, dataquery.ResultFormatTimeSeries)),
|
||||
RefID: "A",
|
||||
TimeRange: timeRange,
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: makeQueryPointer(AzureLogAnalyticsQuery{
|
||||
RefID: "A",
|
||||
ResultFormat: dataquery.ResultFormatTimeSeries,
|
||||
URL: "v1/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/TestDataWorkspace/search",
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
"azureLogAnalytics": {
|
||||
"resources": ["/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/TestDataWorkspace"],
|
||||
"query": "Perf",
|
||||
"resultFormat": "%s",
|
||||
"dashboardTime": true,
|
||||
"timeColumn": "TimeGenerated",
|
||||
"basicLogsQuery": true
|
||||
}
|
||||
}`, dataquery.ResultFormatTimeSeries)),
|
||||
Query: "Perf",
|
||||
Resources: []string{"/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/TestDataWorkspace"},
|
||||
TimeRange: timeRange,
|
||||
QueryType: dataquery.AzureQueryTypeAzureLogAnalytics,
|
||||
AppInsightsQuery: false,
|
||||
DashboardTime: true,
|
||||
BasicLogs: true,
|
||||
TimeColumn: "TimeGenerated",
|
||||
}),
|
||||
Err: require.NoError,
|
||||
},
|
||||
{
|
||||
name: "Basic Logs query with multiple resources",
|
||||
fromAlert: false,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
"azureLogAnalytics": {
|
||||
"resources": ["/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/TestDataWorkspace1", "/subscriptions/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/resourceGroups/cloud-datasources/providers/Microsoft.OperationalInsights/workspaces/TestDataWorkspace2"],
|
||||
"query": "Perf",
|
||||
"resultFormat": "%s",
|
||||
"dashboardTime": true,
|
||||
"timeColumn": "TimeGenerated",
|
||||
"basicLogsQuery": true
|
||||
}
|
||||
}`, dataquery.ResultFormatTimeSeries)),
|
||||
RefID: "A",
|
||||
TimeRange: timeRange,
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: nil,
|
||||
Err: require.Error,
|
||||
},
|
||||
{
|
||||
name: "Basic Logs query with non LA workspace resources",
|
||||
fromAlert: false,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
"azureLogAnalytics": {
|
||||
"resources": ["/subscriptions/test-sub/resourceGroups/test-rg/providers/Microsoft.Insights/components/r1"],
|
||||
"query": "Perf",
|
||||
"resultFormat": "%s",
|
||||
"dashboardTime": true,
|
||||
"timeColumn": "TimeGenerated",
|
||||
"basicLogsQuery": true
|
||||
}
|
||||
}`, dataquery.ResultFormatTimeSeries)),
|
||||
RefID: "A",
|
||||
TimeRange: timeRange,
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: nil,
|
||||
Err: require.Error,
|
||||
},
|
||||
{
|
||||
name: "Basic Logs query from alerts",
|
||||
fromAlert: true,
|
||||
queryModel: backend.DataQuery{
|
||||
JSON: []byte(fmt.Sprintf(`{
|
||||
"queryType": "Azure Log Analytics",
|
||||
"azureLogAnalytics": {
|
||||
"resources": ["/subscriptions/test-sub/resourceGroups/test-rg/providers/Microsoft.Insights/components/r1"],
|
||||
"query": "Perf",
|
||||
"resultFormat": "%s",
|
||||
"dashboardTime": true,
|
||||
"timeColumn": "TimeGenerated",
|
||||
"basicLogsQuery": true
|
||||
}
|
||||
}`, dataquery.ResultFormatTimeSeries)),
|
||||
RefID: "A",
|
||||
TimeRange: timeRange,
|
||||
QueryType: string(dataquery.AzureQueryTypeAzureLogAnalytics),
|
||||
},
|
||||
azureLogAnalyticsQuery: nil,
|
||||
Err: require.Error,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
query, err := buildLogAnalyticsQuery(tt.queryModel, dsInfo, appInsightsRegExp)
|
||||
query, err := buildLogAnalyticsQuery(tt.queryModel, dsInfo, appInsightsRegExp, tt.fromAlert)
|
||||
tt.Err(t, err)
|
||||
if diff := cmp.Diff(&tt.azureLogAnalyticsQuery, query); diff != "" {
|
||||
if diff := cmp.Diff(tt.azureLogAnalyticsQuery, query); diff != "" {
|
||||
t.Errorf("Result mismatch (-want +got): \n%s", diff)
|
||||
}
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user