Commit Graph

5415 Commits

Author SHA1 Message Date
Sofia Papagiannaki 04370f92dc Annotations: Improve get tags query performance (#66182)
* Add benchmark

* Add fix
2023-04-18 13:27:40 +03:00
Ryan McKinley 15b469bd12 UsageStats: Add database age and driver (#66535)
Co-authored-by: Dan Cech <dcech@grafana.com>
2023-04-17 18:27:04 -07:00
Kyle Brandt 2f13c851e4 SSE: (Chore/Instrumentation) Add ds_queries_total metric and move met… (#66695)
* SSE: (Chore/Instrumentation) Add ds_queries_total metric and move metrics to service
2023-04-17 16:12:44 -07:00
George Robinson 19ebb079ba Alerting: Add limits and filters to Prometheus Rules API (#66627)
This commit adds support for limits and filters to the Prometheus Rules
API.

Limits:

It adds a number of limits to the Grafana flavour of the Prometheus Rules
API:

- `limit` limits the maximum number of Rule Groups returned
- `limit_rules` limits the maximum number of rules per Rule Group
- `limit_alerts` limits the maximum number of alerts per rule

It sorts Rule Groups and rules within Rule Groups such that data in the
response is stable across requests. It also returns summaries (totals)
for all Rule Groups, individual Rule Groups and rules.

Filters:

Alerts can be filtered by state with the `state` query string. An example
of an HTTP request asking for just firing alerts might be
`/api/prometheus/grafana/api/v1/rules?state=alerting`.

A request can filter by two or more states by adding additional `state`
query strings to the URL. For example `?state=alerting&state=normal`.

Like the alert list panel, the `firing`, `pending` and `normal` state are
first compared against the state of each alert rule. All other states are
ignored. If the alert rule matches then its alert instances are filtered
against states once more.

Alerts can also be filtered by labels using the `matcher` query string.
Like `state`, multiple matchers can be provided by adding additional
`matcher` query strings to the URL.

The match expression should be parsed using existing regular expression
and sent to the API as URL-encoded JSON in the format:

{
    "name": "test",
    "value": "value1",
    "isRegex": false,
    "isEqual": true
}

The `isRegex` and `isEqual` options work as follows:

| IsEqual | IsRegex  | Operator |
| ------- | -------- | -------- |
| true    | false    |    =     |
| true    | true     |    =~    |
| false   | true     |    !~    |
| false   | false    |    !=    |
2023-04-17 17:45:06 +01:00
Ashley Harrison ab08b4f7f2 Chore: Clean up NavModel interface (#66548)
* clean up navmodel interface

* remove concept of sections from NavModel interface

* clean up applinks
2023-04-17 16:01:32 +01:00
Josh Hunt 6be9264b94 NestedFolders: Don't require dev mode for feature toggle (#66501)
NestedFolders: Promote feature flag to non-dev beta
2023-04-17 15:54:32 +03:00
Serge Zaitsev f58a63b2df Chore: Use time mocks in queryhistory to speed up tests (#66644)
* use time mocks in queryhistory to speed up tests

* remove comments
2023-04-17 12:02:47 +02:00
linoman 4027254b87 Auth: Add SigningKeys Service (#64343)
* Add key service

Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>

* Wire the service

* Rename Service

* Implement GetJWKS

* Slipt interface and implementation

Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>

* Change implementation, add tests

* Align to the expected package hierarchy

* Update CODEOWNERS

* Align names and fix wire.go

* Update pkg/services/signingkeys/signingkeysimpl/service.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* Update pkg/services/signingkeys/signingkeysimpl/service_test.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* Update pkg/services/signingkeys/signingkeysimpl/service_test.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* Update pkg/services/signingkeys/signingkeysimpl/service_test.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* Add AddPrivateKey method to SigningKeysService

* Align tests to the guidelines

* Add test for GetJWKS() method

* Add comments to the interface

* Add FakeSigningKeysService

---------

Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2023-04-17 11:42:37 +02:00
Isabella Siu 1b0cee491a OpenSearch: Add feature toggle for detecting version (#66386)
OpenSearch: Add feature toggle for auto-detecting version
2023-04-14 13:59:31 -04:00
Gabriel MABILLE 3b63844390 RBAC: Feature to override default assignments (#66561)
* RBAC: Feature to override default assignments

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* Add test and trim spaces

* Pass linting

* Apply the rbac overrides to fixed_authentication.config_writer

* Removing from the default ini file for now

* Add grants overrides section to cfg

* slimmer handleGrantOverrides function

---------

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
2023-04-14 17:17:59 +02:00
Ivan Ortega Alba 7bc31ab04b Replace the DS drawer toggle with the new DS picker one (#66551) 2023-04-14 13:01:10 +02:00
Ashley Harrison ee166205cd Navigation: move connections in IA (#66559)
move connections in IA
2023-04-14 11:46:18 +01:00
Piotr Jamróz 1855bca2fa Explore: Promote exploreMixedDatasource feature toggle to beta (#66552)
Promote exploreMixedDatasource to beta
2023-04-14 13:07:46 +03:00
Ivana Huckova f48c858ca2 Elasticsearch: Change feature toggle for backend migration to enableElasticsearchBackendQuerying (#66500)
* Elasticsearch: Change feature toggle for backend migration to enableElasticsearchBackendQuerying

* Update logic in data source

* Run test to re-genartate toggle
2023-04-14 12:24:35 +03:00
Arati R cab3ba519a NestedFolders: Add folder service registry with dashboard service implementation (#65033)
* Delete folders, dashboards with registry service
Co-authored-by: Serge Zaitsev <hello@zserge.com>
* Update signature of ProvideDashboardServiceImpl
* Regenerate mockery file
* Add test for DeleteInFolder
* Add test for DeleteDashboardsInFolder
* Delete child dashboard associations via registry
* Add validation of folder uid and org id

---------

Co-authored-by: Serge Zaitsev <hello@zserge.com>
2023-04-14 11:17:23 +02:00
Ashley Harrison 4abe0249ba Chore: Clean up old navigation (#66287)
* remove code outside of the topnav feature flag

* delete NavBar folder

* remove topnav toggle from backend

* restructure AppChrome folder

* fix utils mock

* fix applinks tests

* remove tests since they're covered in e2e

* fix 1 of the approotpage tests

* Fix another dashboardpage test

* remove reverse portalling + test for plugins using deprecated onNavChanged method

* kick drone

* handle correlations
2023-04-14 09:43:11 +01:00
Ryan McKinley fef490e4f8 Analytics: Add total dashboard byte size to reporting (#66425)
Co-authored-by: Dan Cech <dcech@grafana.com>
2023-04-13 21:37:03 +03:00
Artur Wierzbicki 2136e680c4 Previews: remove dashboard previews backend (#66176)
* remove dashboard previews backend

* remove dashboard previews backend

* bring back the migration

* bring back the migration

* bring back the migration
2023-04-13 21:42:24 +04:00
Yuri Tseretyan afd52d0866 Alerting: use alerting GrafanaReceiver and BuildReceiverConfiguration in Grafana (#65224)
* replace receiver errors with one from alerting
* add the converter to alerting models
* update buildReceiverIntegration to accept GrafanaReceiver
---------

Co-authored-by: George Robinson <george.robinson@grafana.com>
2023-04-13 12:25:32 -04:00
Alexander Zobnin 7476219b0c SAML: Configuration UI (#64054)
* Add initial authentication config page skeleton

* Add initial SAML config page WIP

* Add few more pages

* Add connect to IdP page

* Assertion mappings page stub and url params

* Able to save settings

* Some tweaks for authentication page

* Tweak behaviour

* Tweak provider name

* Move SAML config pages to enterprise

* minor refactor

* Able to reset settings

* Configure key and cert from UI

* Refactor WIP

* Tweak styles

* Optional save button

* Some tweaks for the page

* Don't show info popup when save settings

* Improve key/cert validation

* Fetch provider status and display on auth page

* Add settings list to the auth page

* Show call to action card if no auth configured

* clean up

* Show authentication page only if SAML available

* Add access control for SSO config page

* Add feature toggle for auth config UI

* Add code owners for auth config page

* Auth config UI disabled by default

* Fix feature toggle check

* Apply suggestions from review

* Refactor: use forms for steps

* Clean up

* Improve authentication page loading

* Fix CTA link

* Minor tweaks

* Fix page route

* Fix formatting

* Fix generated code formatting
2023-04-13 16:07:43 +02:00
juanicabanas 9964441b80 PublicDashboards: Code formatted (#66462) 2023-04-13 16:29:09 +03:00
Gábor Farkas b928fce070 Loki: Remove alpha feature toggle lokiDataframeApi (#65918)
loki: remove experimental feature-flag
2023-04-13 15:22:09 +02:00
Gábor Farkas 531caec602 Loki: Add feature flag to enable dataplane-compliant metric frames (#66017)
* loki: add feature flag to enable dataplane-compatible metric frames

* linter fix
2023-04-13 15:07:08 +02:00
gotjosh 2bbf0c9de4 Alerting: Allow Rules to Schedule to be filtered by Rule Group (#59990)
* Alerting: Allow Rules to Schedule to be filtered by Rule Group
2023-04-13 12:55:42 +01:00
Misi 4cac95d1c2 Auth: Remove the session cookie only if it's invalid or revoked (#65984)
Remove the cookie if it's invalid or revoked
2023-04-13 09:23:39 +02:00
Michael Mandrus 5626461b3c Caching: Refactor enterprise query caching middleware to a wire service (#65616)
* define initial service and add to wire

* update caching service interface

* add skipQueryCache header handler and update metrics query function to use it

* add caching service as a dependency to query service

* working caching impl

* propagate cache status to frontend in response

* beginning of improvements suggested by Lean - separate caching logic from query logic.

* more changes to simplify query function

* Decided to revert renaming of function

* Remove error status from cache request

* add extra documentation

* Move query caching duration metric to query package

* add a little bit of documentation

* wip: convert resource caching

* Change return type of query service QueryData to a QueryDataResponse with Headers

* update codeowners

* change X-Cache value to const

* use resource caching in endpoint handlers

* write resource headers to response even if it's not a cache hit

* fix panic caused by lack of nil check

* update unit test

* remove NONE header - shouldn't show up in OSS

* Convert everything to use the plugin middleware

* revert a few more things

* clean up unused vars

* start reverting resource caching, start to implement in plugin middleware

* revert more, fix typo

* Update caching interfaces - resource caching now has a separate cache method

* continue wiring up new resource caching conventions - still in progress

* add more safety to implementation

* remove some unused objects

* remove some code that I left in by accident

* add some comments, fix codeowners, fix duplicate registration

* fix source of panic in resource middleware

* Update client decorator test to provide an empty response object

* create tests for caching middleware

* fix unit test

* Update pkg/services/caching/service.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* improve error message in error log

* quick docs update

* Remove use of mockery. Update return signature to return an explicit hit/miss bool

* create unit test for empty request context

* rename caching metrics to make it clear they pertain to caching

* Update pkg/services/pluginsintegration/clientmiddleware/caching_middleware.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Add clarifying comments to cache skip middleware func

* Add comment pointing to the resource cache update call

* fix unit tests (missing dependency)

* try to fix mystery syntax error

* fix a panic

* Caching: Introduce feature toggle to caching service refactor (#66323)

* introduce new feature toggle

* hide calls to new service behind a feature flag

* remove licensing flag from toggle (misunderstood what it was for)

* fix unit tests

* rerun toggle gen

---------

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2023-04-12 12:30:33 -04:00
Kyle Brandt e78be44e1a SSE: Dataplane Compliance (#65927)
Takes a specific code path for data that identifies itself as dataplane instead of "guessing" what the data is.

The data must identify itself by being in the dataplane by having both the following frame metadata properties:

- TypeVersion property that is greater than 0.0
- 'Type' property

The flag is disableSSEDataplane and disables this functionality and uses the old code for all queries regardless.

See https://github.com/grafana/grafana-plugin-sdk-go/blob/main/data/contract_docs/contract.md for dataplane details.
2023-04-12 12:24:34 -04:00
ismail simsek 08727b7d6c Prometheus: Show initial hint on builder mode when metric lookup disabled (#65827)
* Show initial hint on builder mode when metric lookup disabled

* Disable MetricEncyclopedia and label request when metric lookup is disabled
2023-04-12 17:22:35 +02:00
Emil Tullstedt e0385d08a8 Correlations: Update feature toggle status to beta (#66390) 2023-04-12 17:56:06 +03:00
Ezequiel Victorero 3224b4c8f0 PublicDashboards: validate only one public dashboard per dashboard (#66228) 2023-04-12 10:57:42 -03:00
Ivana Huckova 303777e682 Elasticsearch: Add feature toggle to disable running queries trough backend (#66260)
* Elasticsearch: Create feature toggle to disable running queries trough backend

* Fix frontend tests running trough frontend

* Update description of feature toggle
2023-04-12 15:20:43 +03:00
juanicabanas 92e591d2e1 PublicDashboards: metrics collected in background service (#65836) 2023-04-11 14:36:50 -03:00
Gabriel MABILLE 5df9e64986 AuthN: feature flag for external service authentication (#66160)
* AuthN: feature flag for external service authentication

* Change the description message
2023-04-11 15:21:19 +02:00
Ashley Harrison fda38de293 Library panels: Ensure pagination controls are always correctly displayed (#65283)
correctly count library panels in the general folder
2023-04-11 14:16:20 +01:00
Piotr Jamróz b68be999f7 Correlations: Add usage stats about correlations (#66021)
* Add usage stats about correlations

* Add stats.correlations.count to collected stats

* Expose grafana_stat_totals_correlations metric

* Organize imports
2023-04-11 08:53:34 +02:00
Ryan McKinley af31c77331 Dataplane: Support prometheus dataplane contract for transformations and name matchers (#65237)
Co-authored-by: Brendan O'Handley <brendan.ohandley@grafana.com>
2023-04-08 00:13:19 +03:00
Stephanie Hingtgen 536fad5b61 Plugins: add env variables needed for the secure socks proxy in the plugins sdk (#65878) 2023-04-07 08:20:25 -06:00
Matthew Jacobson 63187fae0c Alerting: Remove and revert flag alertingBigTransactions (#65976)
* Alerting: Remove and revert flag alertingBigTransactions

This is a partial revert of #56575 and a removal of the `alertingBigTransactions` flag.

Real-word use has seen no clear performance incentive to maintain this flag. Lowered db connection count
came at the cost of significant increase in CPU usage and query latency.

* Fix lint backend

* Removed last bits of alertingBigTransactions

---------

Co-authored-by: Armand Grillet <2117580+armandgrillet@users.noreply.github.com>
2023-04-06 18:06:25 +02:00
gotjosh 1c3ce0735f Alerting: Tiny refactor on the eval and schedule packages (#66130)
* Alerting: Tiny refactor on the eval and schedule packages

two very small things:

- We had a constructor on something called a `Context` which is not a `context.Context` so let's just name that constructor `NewContext`
- The user that we use to run query evaluations is the same (with some variation) abstract it to a function so that it can be re-used when necessary.

* Update pkg/services/ngalert/schedule/schedule.go

Co-authored-by: Alexander Weaver <weaver.alex.d@gmail.com>

* Update pkg/services/ngalert/schedule/schedule.go

Co-authored-by: Alexander Weaver <weaver.alex.d@gmail.com>

---------

Co-authored-by: Alexander Weaver <weaver.alex.d@gmail.com>
2023-04-06 16:02:28 +01:00
linoman 3e85c90759 Restructure the org upsert method (#64763) 2023-04-06 16:48:31 +02:00
Sofia Papagiannaki 988a120d6d Search v1: Add support for inherited folder permissions if nested folders are enabled (#63275)
* Add features dependency to SQLBuilder

* Add features dependency to AccessControlDashboardPermissionFilter

* Add test for folder inheritance

* Dashboard permissions: Return recursive query

* Recursive query for inherited folders

* Modify search builder

* Adjust db.SQLBuilder

* Pass flag to SQLbuilder if CTEs are supported

* Add support for mysql < 8.0

* Add benchmarking for search with nested folders

* Set features to AlertStore

* Update pkg/infra/db/sqlbuilder.go

Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>

* Set features to LibraryElementService

* SQLBuilder tests with nested folder flag set

* Apply suggestion from code review

Co-authored-by: IevaVasiljeva <ieva.vasiljeva@grafana.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2023-04-06 11:16:15 +03:00
Matthew Jacobson 85f738cdf9 Alerting: Add endpoint to revert to a previous alertmanager configuration (#65751)
* Alerting: Add endpoint to revert to a previous alertmanager configuration

This endpoint is meant to be used in conjunction with /api/alertmanager/grafana/config/history to
revert to a previously applied alertmanager configuration. This is done by ID instead of raw config
string in order to avoid secure field complications.
2023-04-05 14:10:03 -04:00
Ezequiel Victorero 232834f455 PublicDashboards: fix collapsed rows queries (#66014) 2023-04-05 10:35:00 -03:00
Andrej Ocenas 37eaf50197 FlameGraph: Use pyroscope/flamegraph (#65896) 2023-04-05 14:57:24 +02:00
Will Browne bff9f4c890 Plugins: Move config factory to pluginsintegration package (#65716)
* move config to pluginsintegration package

* change to all plugin toggle

* fixes

* fixes

* fix lerna

* fix test
2023-04-05 14:40:08 +02:00
Sofia Papagiannaki caac9838d8 Build: Fix Redis/Memcached integration tests (#64298)
* Build: Fix integration cache tests

* Allow REDIS_URL with scheme

* Reduce cache integration tests timeout to 5m

* Apply suggestion from code review

* Run redis/memcached integration tests in OSS pipelines

* Change redis image
2023-04-05 11:55:55 +03:00
Gábor Farkas 4ded937c79 Show traceids for failing and successful requests (#64903)
* tracing: show backend trace ids in frontend

* better trace id naming

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* better trace id naming

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* better trace id naming

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* added feature flag

* bind functionality to the feature flag

* use non-generic name for traceid header

* fixed tests

* loki: do not create empty fields

* do not add empty fields

* fixed graphite test mock data

* added unit-tests to queryResponse

* added unit-tests for backend_srv

* more typescript-friendly check

* added unit-tests for runRequest

---------

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
2023-04-05 09:13:24 +02:00
Alexander Weaver fb520edd72 Alerting: Use a completely isolated context for state history writes (#64989)
* Add fresh context with timeout and same log properties, re-derive logger

* Unify timeout constants

* Move ctx after shortcut that got added through rebasing

* Unify timeouts

* Port opentracing's SpanFromContext and ContextFromSpan to the grafana tracing package

* Support both opentracing and otel variants

* Better document why we're creating a new ctx

* Add new func to FakeSpan which was added after rebase

* Support grafana-specific traceID key in both tracer implementations
2023-04-04 16:41:46 -05:00
gotjosh 673aee30c2 Feature Flags: Remove dev-mode and alpha from grpc server feature flag (#65789)
* Feature Flags: Remove dev-mode from grpc server feature flag

---------

Co-authored-by: Todd Treece <todd.treece@grafana.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-04-04 08:42:07 +01:00
Sofia Papagiannaki a270188f0c Nested folders: Fix moving folder under root (#65684)
* Nested folders: Fix moving folder under root

* Add store test for not empty parent after update

* Modify folder and document store update implementation

Move folder only if NewParentUID is not nil

* Apply suggestion from code review
2023-04-03 21:24:36 +03:00