Unified Storage: Remove wildcard queries (#101441)

* WIP adding custom analyzer so we can do substring search efficiently

* Adding unit tests for title search

* formatting

* adds more title search unit tests

* organize helpers

* fixes issue caused by having two title mappings

* Removes camelcase token filter since it prevents you from searching for a substring of chars and numbers. Adds regression test.

* adds back mapping for title_phrase

* use simple analyzer for input query string so it doesn't filter out english stop words

* ran bleve tests, table snapshots updated

* ignore linter for "unused" test functions. They are very helpful for troubleshooting search. Keeping them.

* only log total hits and query cost if result not nil

* fixes failing test - one more field because there are two title mappings now

* fix test

* fixes test - only take first item when its the title

* Adds separate internal field for title ngram mapping.

When searching with a query, results are sorted by score desc.

When searching without a query, results are sorted by title desc.

Adjusts ngram max to be 10.

Text queries are a disjunction of an exact match, phrase match, and a match. Boosted to have priority in that order.

Adds more unit tests for searching.

* linter

* fix test

* ran tests - generated new test dash json

* sort by title phrase instead of title

* fix test - not relying on /apis/dashboard/search to apply title sorting anymore
This commit is contained in:
owensmallwood
2025-03-07 09:51:15 -06:00
committed by GitHub
parent da8f26a07c
commit 827da46c51
14 changed files with 561 additions and 31 deletions
+6 -1
View File
@@ -62,6 +62,9 @@ type IndexableDocument struct {
// The generic display name
Title string `json:"title,omitempty"`
// internal field for searching title with ngrams
TitleNgram string `json:"title_ngram,omitempty"`
// internal sort field for title ( don't set this directly )
TitlePhrase string `json:"title_phrase,omitempty"`
@@ -169,7 +172,8 @@ func NewIndexableDocument(key *ResourceKey, rv int64, obj utils.GrafanaMetaAcces
Key: key,
RV: rv,
Name: key.Name,
Title: title, // We always want *something* to display
Title: title, // We always want *something* to display
TitleNgram: title,
TitlePhrase: strings.ToLower(title), // Lowercase for case-insensitive sorting
Labels: obj.GetLabels(),
Folder: obj.GetFolder(),
@@ -264,6 +268,7 @@ const SEARCH_FIELD_NAMESPACE = "namespace"
const SEARCH_FIELD_NAME = "name"
const SEARCH_FIELD_RV = "rv"
const SEARCH_FIELD_TITLE = "title"
const SEARCH_FIELD_TITLE_NGRAM = "title_ngram"
const SEARCH_FIELD_TITLE_PHRASE = "title_phrase" // filtering/sorting on title by full phrase
const SEARCH_FIELD_DESCRIPTION = "description"
const SEARCH_FIELD_TAGS = "tags"