Commit graph

16 commits

Author SHA1 Message Date
Dhanu Saputra ad4dc380d5
track-number-of-optimized-regexp-label-matchers - make isRegexOptimized (#481)
public

Signed-off-by: dhanu <andreasdhanu@gmail.com>
2023-04-18 07:24:41 +00:00
Marco Pracucci c461e22341
Improve fast regexp matcher cache (#482)
* Limit FastRegexMatcher by size (bytes) and add a TTL to each entry

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Add TestNewFastRegexMatcher_CacheSizeLimit

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Tolerate ristretto goroutines when checking goroutine leaks

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Tolerate ristretto goroutines when checking goroutine leaks

Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-04-17 15:20:58 +02:00
Patrick Oyarzun ae170f644c
Optimize long alternate lists (#463)
* Use a prefix trie for long alternate lists

* Add test for non terminal node

* Fix panic in FuzzFastRegexMatcher_WithFuzzyRegularExpressions when the fuzzy regex is invalid

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Address PR feedback

* Update model/labels/regexp_test.go

Co-authored-by: Marco Pracucci <marco@pracucci.com>

* Replace trie with slice or map depending on input size

* Fix tests

* Pull in tests from @pracucci's branch

* Add setMatches back in

* Use stringMatcher when it's faster

* Fix linter

* Estimate alternates ahead of time

* Simplify construction with `IndexByte`

* Add test and early return for empty regexp.

* Fix race conditions in tests

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Marco Pracucci <marco@pracucci.com>
2023-04-01 08:35:35 +02:00
Marco Pracucci 9fe1bd2d63
Improve TestAnalyzeRealQueries (#464)
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-31 08:27:43 +00:00
Marco Pracucci 05a3a79015
Cache optimized regexp matchers (#465)
* Cache optimized regexp matchers

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Added BenchmarkNewFastRegexMatcher_CacheMisses

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Improved benchmark

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Improved benchmark

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Use LRU cache v2

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Run gofumpt

Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-31 04:05:26 +02:00
Marco Pracucci 242e82b8e6
Optimize regex star operation (#448)
* Optimize .* regex matcher

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Consistent benchmark runs for BenchmarkFastRegexMatcher

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Fixed TestParseExpressions

Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-09 09:38:41 +01:00
Marco Pracucci 1e7ad0ec11
Optimized very long case insensitive alternations (#444)
* Optimized very long case insensitive alternations

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Run common regexps in BenchmarkFastRegexMatcher

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Modify BenchmarkNewFastRegexMatcher to benchmark the NewFastRegexMatcher() function

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Reduced allocations by optimizeEqualStringMatchers()

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Fixed typo in comments

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Fixed typo in test case name

Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-02 17:20:52 +01:00
Marco Pracucci 383ea59ce1
Add TestAnalyzeRealQueries (#443)
* Add TestAnalyzeRealQueries

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Add nolint directive

Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-01 15:50:04 +01:00
Marco Pracucci eeecfee885
Do not optimize regexps with begin/end text anchors inside (#433)
* Do not optimize regexps with being/end text anchors inside

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Explicit case for begin/end text in stringMatcherFromRegexpInternal()

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Added more test cases

Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-01 14:50:26 +01:00
Marco Pracucci 2e0ecc013f
Fix containsStringMatcher() when the text contains multiple occurrences of a substring (#431)
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-01 11:18:30 +00:00
Marco Pracucci c77900d58e
Optimized FastRegexMatcher when the regex contains a case insensitive alternation made with concats too (#430)
* Optimized FastRegexMatcher when the regex contains a case insensitive alternation made with concats too

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Do not use a pointer to hold whether the matches are case sensitive

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Improved unit tests based on review feedback

Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-01 10:49:25 +01:00
Jesus Vazquez 51f26e0cba Necessary changes to make the merge work
This commit disables some unused workflows on our CI.

Also uses grafana/regexp instead of regexp which is blackisted.

Also updates head_test TestHeadReadWriterRepair increasing
ChunkWriteQueueSize to 1 so that the chunk disk mapper uses the async
queue. This seems to be default behaviour in upstream prometheus and
without this option our test fails.
2022-04-13 14:45:43 +02:00
Jesus Vazquez 48aa5cd096 Merge remote-tracking branch 'upstream/main' into jvp/merge-prometheus-main 2022-04-12 16:40:00 +02:00
Bryan Boreham 579331446a
Allow downstream projects to use faster regexp (#10251)
* Add benchmark for FastRegexMatcher

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Use modified regexp package with optimisations

See https://github.com/grafana/regexp/tree/speedup#readme

Includes the following changes proposed upstream:
* [regexp: allow patterns with no alternates to be one-pass](https://go-review.googlesource.com/c/go/+/353711)
* [regexp: speed up onepass prefix check](https://go-review.googlesource.com/c/go/+/354909)
* [regexp: handle prefix string with fold-case](https://go-review.googlesource.com/c/go/+/358756)
* [regexp: avoid copying each instruction executed](https://go-review.googlesource.com/c/go/+/355789)
* [regexp: allow prefix string anchored at beginning](https://go-review.googlesource.com/c/go/+/377294)

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Use regexp code identical to upstream Go

Change `grafana/regexp` import to use `main` branch.

This means Prometheus is not using the proposed optimisations, but
downstream users of Prometheus code are able to `replace` the library
with the `speedup` branch which does.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-02-08 11:03:20 +01:00
Peter Štibraný 294c155bb6 Merge remote-tracking branch 'upstream/main' into merge-upstream 2021-11-18 15:48:40 +01:00
beorn7 c954cd9d1d Move packages out of deprecated pkg directory
This creates a new `model` directory and moves all data-model related
packages over there:
  exemplar labels relabel rulefmt textparse timestamp value

All the others are more or less utilities and have been moved to `util`:
  gate logging modetimevfs pool runtime

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-09 08:03:10 +01:00
Renamed from pkg/labels/regexp_test.go (Browse further)