Commit graph

29 commits

Author SHA1 Message Date
naivewong 13c80a5979 Optimize queries using regex matchers for set lookups (#602)
* Original version of the set optimization

Signed-off-by: naivewong <867245430@qq.com>

* simple set matcher

Signed-off-by: naivewong <867245430@qq.com>

* simple set matcher

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* add benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* use genSeries from #467

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>
2019-05-27 16:54:46 +05:30
Brian Brazil 6ac81cc7a9
Correctly handle empty labels. (#594)
Currently a time series with empty labels is not treated the same
as one with missing labels. Currently this can only come from
ALERTS&ALERT_FOR_STATE so it's unlikely anyone has actually hit it.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-05-07 11:00:16 +01:00
Krasi Georgiev 288f67efbf
fix some unchecked errors and remove unused vars. (#592)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-04-25 13:07:04 +03:00
Brian Brazil 259847a6b1
Be smarter in how we look at matchers. (#572)
* Add unittests for PostingsForMatcher.

* Selector methods are all stateless, don't need a reference.

* Be smarter in how we look at matchers.

Look at all matchers to see if a label can be empty.

Optimise Not handling, so i!="2" is a simple lookup
rather than an inverse postings list.

All all the Withouts together, rather than
having to subtract each from all postings.

Change the pre-expand the postings logic to always do it before doing a
Without only. Don't do that if it's already a list.

The initial goal here was that the oft-seen pattern
i=~"something.+",i!="foo",i!="bar" becomes more efficient.

benchmark                                                            old ns/op     new ns/op     delta
BenchmarkHeadPostingForMatchers/n="1"-4                              5888          6160          +4.62%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4                      7190          6640          -7.65%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4                      6038          5923          -1.90%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4                     6030884       4850525       -19.57%
BenchmarkHeadPostingForMatchers/i=~".*"-4                            887377940     230329137     -74.04%
BenchmarkHeadPostingForMatchers/i=~".+"-4                            490316101     319931758     -34.75%
BenchmarkHeadPostingForMatchers/i=~""-4                              594961991     130279313     -78.10%
BenchmarkHeadPostingForMatchers/i!=""-4                              537542388     318751015     -40.70%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4              10460243      8565195       -18.12%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4       44964267      8561546       -80.96%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4                42244885      29137737      -31.03%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4              35285834      32774584      -7.12%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4             8951047       8379024       -6.39%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4       63813335      30672688      -51.93%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4     45381112      44924397      -1.01%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-09 11:59:45 +01:00
zhulongcheng e11e01f68d Fix exported function comments (#566)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-25 10:17:28 +02:00
Krasi Georgiev 48c439d26d
fix statick check errors (#475)
fix the tests for `check_license` and `staticcheck`

the static check also found some actual bugs.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-02 19:48:42 +03:00
Krasi Georgiev 090b6852e1
remove unused PrefixMatcher (#474)
* remove unused `PrefixMatcher`

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-12-28 21:13:02 +03:00
nilsocket 80981a6aac FromMap(), sorts and returns instead of calling New() (#433)
Signed-off-by: nilsocket <nilsocket@gmail.com>
2018-11-14 13:43:03 +01:00
Ben a8351dc9d0 Using filepath.Join() instead of strings with slashes (#428)
fixes: https://github.com/prometheus/tsdb/issues/426
Using `filepath.Join()` instead of strings containing forward slash path delimiters (needed for non-*nix OSes), as suggested by @krasi-georgiev
2018-10-25 10:32:57 +01:00
kun 5f929254a3 Fix labels bench test
Signed-off-by: kun <oiooj@qq.com>
2018-03-22 12:28:09 +08:00
Fabian Reinartz adb6be73dd move label read function int labels pkg 2017-12-21 11:55:58 +01:00
Goutham Veeramachaneni 239cbae154
Merge pull request #228 from Gouthamve/not-matchers
Select series with label unset for != and !~
2017-12-21 12:10:51 +05:30
Goutham Veeramachaneni 3158b03e6c Select series with label unset for != and !~
Fixes https://github.com/prometheus/prometheus/issues/3575

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2017-12-19 02:47:31 +00:00
Callum Styan bcde61b237 we shouldn't assume we'll receive the labels in sorted order 2017-12-18 15:32:39 -08:00
Callum Styan 8326e410d0 decouple tsdb main.go (and tests) from prometheus/pkg 2017-12-18 15:26:16 -08:00
Callum Styan ef99a94ffe use test utils in labels_test 2017-12-06 17:13:00 -08:00
Fabian Reinartz c36d574290 Replace single head lock with granular locks
This adds various new locks to replace the single big lock on
the head. All parts now must be COW as they may be held by clients
after initial retrieval.
Series by ID and hashes are now held in a stripe lock to reduce
contention and total holding time during GC. This should reduce
starvation of readers.
2017-09-05 14:41:39 +02:00
Dmitry Ilyevsky 37194b7a30 Add prefix label matcher.
Implement labels.PrefixMatcher and use interface conversion in querier
to optimize label tuples search.

[unit-tests]: Fix bug and populate label index for mock index.

Signed-off-by: Dmitry Ilyevsky <ilyevsky@gmail.com>
2017-07-22 01:06:30 -07:00
Julius Volz a25f874ec4 Simplify Labels.Equals()
Also extend the Compare() tests to also test Labels.Equals().
2017-04-28 16:01:37 +02:00
Fabian Reinartz 778103b450 Add liecence file and headers 2017-04-10 20:59:45 +02:00
Fabian Reinartz 7de2217011 Add fast-path for equality matching 2017-04-05 15:37:48 +02:00
Goutham Veeramachaneni 761e4768f3
Lint and Vet Fixes 2017-03-19 21:35:01 +05:30
Fabian Reinartz 51ea34d381 Fix label sorting benchmark 2017-03-17 10:07:10 +01:00
Fabian Reinartz eedbebe1d7 Misc cleanups 2017-03-06 14:34:49 +01:00
Fabian Reinartz 286293802b Add merge postings 2016-12-28 11:02:19 +01:00
Fabian Reinartz 5efe1d178e labels: add Compare and String methods 2016-12-24 13:49:35 +01:00
Fabian Reinartz 0d0c5cfaf1 labels: add string constructor, expose matcher 2016-12-24 10:19:46 +01:00
Fabian Reinartz 0a94f58f1a Fix test import of labels, simplify constructor names 2016-12-21 15:12:26 +01:00
Fabian Reinartz ede733ab6c Extract labels package 2016-12-21 09:39:01 +01:00