mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-11-14 17:44:06 -08:00
Code Issues Actions 8 Packages Projects Releases Wiki Activity
9321 commits 248 branches 440 tags 657 MiB
Commit graph

9321 commits

This branch
This branch
All branches
Author SHA1 Message Date
Bryan Boreham c2efd5bf96 promql: copy data when short-circuiting (#9552) 
* promql: copy data when short-circuiting

Because the range query loop re-uses the output buffer each time round,
we must copy results into the buffer rather than using input as output.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
(cherry picked from commit a278ea4b58)
 2021-10-26 23:17:28 +02:00
Marco Pracucci fbd1b4fb84
Merge pull request #21 from grafana/reuse-expand-postings-method-in-new-postings-cloner 
Use `ExpandPostings()` in `NewPostingsCloner()`
 2021-10-13 17:01:24 +02:00
Oleg Zaytsev 14cb8d8696
Use ExpandPostings() in NewPostingsCloner() 
We should reuse that method, not only because it saves code, but also
because that method is a good candidate to use a pool of slices some
day.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
 2021-10-13 15:26:36 +02:00
Marco Pracucci 8bc1b41795
Merge pull request #20 from grafana/test-approxBytesPerEntry-is-correct 
Test that approxBytesPerEntry const is correct
 2021-10-11 15:42:38 +02:00
Oleg Zaytsev 3dd18a1ece
Test that approxBytesPerEntry const is correct 
The comment on the constant mentions where it comes from, so we can
actually check its value in that test.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
 2021-10-11 15:15:56 +02:00
Cyril Tovena 62935a1241
Merge pull request #19 from grafana/regexpopti 
Optimize most common regexp
 2021-10-11 14:15:51 +02:00
Cyril Tovena 01187fbce2
Review feedback 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-11 13:59:24 +02:00
Cyril Tovena c8dfba9fa4
Add better comments 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-11 13:56:22 +02:00
Cyril Tovena d41e470693
Update pkg/labels/regexp.go 
Co-authored-by: Marco Pracucci <marco@pracucci.com>
 2021-10-11 11:13:53 +02:00
Cyril Tovena 37416bf907
Update pkg/labels/regexp.go 
Co-authored-by: Marco Pracucci <marco@pracucci.com>
 2021-10-11 11:09:55 +02:00
Cyril Tovena 5c5b38712f
Add more comments. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-11 10:10:51 +02:00
Cyril Tovena 9c33f392e6
Better benchmarks values. 
```
❯ benchcmp  before.txt after.txt
benchmark                                                           old ns/op     new ns/op     delta
BenchmarkNewFastRegexMatcher/(foo|bar)-16                           2487          2520          +1.33%
BenchmarkNewFastRegexMatcher/foo.*-16                               8856          4531          -48.84%
BenchmarkNewFastRegexMatcher/.*foo-16                               25195         6389          -74.64%
BenchmarkNewFastRegexMatcher/^.*foo$-16                             25626         6253          -75.60%
BenchmarkNewFastRegexMatcher/^.+foo$-16                             25429         6248          -75.43%
BenchmarkNewFastRegexMatcher/.*-16                                  429707        8439          -98.04%
BenchmarkNewFastRegexMatcher/.+-16                                  380165        8503          -97.76%
BenchmarkNewFastRegexMatcher/foo.+-16                               8180          4586          -43.94%
BenchmarkNewFastRegexMatcher/.+foo-16                               25214         6255          -75.19%
BenchmarkNewFastRegexMatcher/foo_.+-16                              8116          4334          -46.60%
BenchmarkNewFastRegexMatcher/foo_.*-16                              8354          4287          -48.68%
BenchmarkNewFastRegexMatcher/.*foo.*-16                             206076        19227         -90.67%
BenchmarkNewFastRegexMatcher/.+foo.+-16                             208434        18793         -90.98%
BenchmarkNewFastRegexMatcher/#00-16                                 33045         3936          -88.09%
BenchmarkNewFastRegexMatcher/(?s:.*)-16                             403806        4208          -98.96%
BenchmarkNewFastRegexMatcher/(?s:.+)-16                             418177        4150          -99.01%
BenchmarkNewFastRegexMatcher/(?s:^.*foo$)-16                        24452         5661          -76.85%
BenchmarkNewFastRegexMatcher/^(?i:foo|oo)|(bar)$-16                 48087         23183         -51.79%
BenchmarkNewFastRegexMatcher/((.*)(bar|b|buzz)(.+)|foo)$-16         645430        47193         -92.69%
BenchmarkNewFastRegexMatcher/^$-16                                  37244         3912          -89.50%
BenchmarkNewFastRegexMatcher/(prometheus|api_prom)_api_v1_.+-16     17205         10006         -41.84%
BenchmarkNewFastRegexMatcher/10\.0\.(1|2)\.+-16                     6776          7011          +3.47%
BenchmarkNewFastRegexMatcher/10\.0\.(1|2).+-16                      14792         4674          -68.40%
BenchmarkNewFastRegexMatcher/((fo(bar))|.+foo)-16                   497007        17410         -96.50%
```

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-11 09:45:02 +02:00
Cyril Tovena 715274ec9d
Reintroduce tests. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-08 10:10:18 +02:00
Cyril Tovena 7dab60e56d
More optimization. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-07 17:15:20 +02:00
Christian Simon 50c1060328
Implement PostingsCloner and implement promise/future pattern for PostingsForMatchers (#14) 
* Fix tests not closing head/block properly

Close Head properly in TestWalRepair_DecodingError
Close Block properly in  TestReadIndexFormatV1

Co-authored-by: Christian Simon <simon@swine.de>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Implement PostingsCloner

PostingsCloner allows obtaining independend clones of a Postings, that
can be used for subsequent calls.

Co-authored-by: Christian Simon <simon@swine.de>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* IndexReader.PostingsForMatchers() and implement Cache

IndexReader now offers the PostingsForMatchers functionality, which is
provided by PostingsForMatchersCache.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>

Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>
 2021-10-07 15:16:09 +02:00
Cyril Tovena 852b553c00
Fixes the code and improve tests. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-07 15:10:26 +02:00
Cyril Tovena 9173cade01
Merge pull request #15 from grafana/improvesetmatches 
Improve `findSetMatches `to support concatenation.
 2021-10-07 14:48:53 +02:00
Marco Pracucci 23ac1d73ef
Fixed timeout increase 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
 2021-10-07 14:35:55 +02:00
Marco Pracucci 7aea989fad
Increase tests timeout 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
 2021-10-07 14:26:46 +02:00
Cyril Tovena 3c13a5e9bd
Improve tests and bench. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-07 14:25:31 +02:00
Cyril Tovena 54b3371060
Merge branch 'improvesetmatches' into regexpopti 2021-10-07 14:05:14 +02:00
Cyril Tovena ce28912439
Fixes a test now that we support more setMatches. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-07 14:04:52 +02:00
Cyril Tovena 2d879c65fc
Avoid API break 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-07 14:00:16 +02:00
Cyril Tovena 74c5667b63
implement optimization. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-07 13:56:31 +02:00
Cyril Tovena f7f80a3abf
Update pkg/labels/regexp_test.go 
Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>
 2021-10-06 16:44:26 +02:00
Cyril Tovena a1f6c747ed
Merge branch 'improvesetmatches' into regexpopti 2021-10-06 16:36:29 +02:00
Cyril Tovena 969497c5a3
more review feedback 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-06 16:33:49 +02:00
Cyril Tovena 9733377257
Review feedback 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-06 16:29:16 +02:00
Cyril Tovena f32ec95722
Add more tests and support begin/end text. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-06 16:22:48 +02:00
Cyril Tovena e9f0f0ebed
Improves test and benchmark 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-06 15:24:57 +02:00
Peter Štibraný b3ae917f96
Merge pull request #17 from grafana/symbols-cleanup 
Splitting compactor: remove unused symbols from index of compacted block
 2021-10-06 14:36:45 +02:00
Cyril Tovena 92736182ce
Adding benchmark. 
```
benchstat before.txt after.txt
name                                                      old time/op    new time/op    delta
PostingsForMatchers/Head/n="1"-16                            480ns ± 4%     469ns ± 4%      ~     (p=0.095 n=5+5)
PostingsForMatchers/Head/n="1",j="foo"-16                    720ns ± 1%     705ns ± 2%      ~     (p=0.056 n=5+5)
PostingsForMatchers/Head/j="foo",n="1"-16                    712ns ± 3%     726ns ± 7%      ~     (p=0.841 n=5+5)
PostingsForMatchers/Head/n="1",j!="foo"-16                   833ns ± 3%     827ns ± 1%      ~     (p=0.548 n=5+5)
PostingsForMatchers/Head/i=~"1[0-9]",j=~"foo|bar"-16        6.66ms ± 4%    0.00ms ± 1%   -99.97%  (p=0.008 n=5+5)
PostingsForMatchers/Head/j=~"foo|bar"-16                     897ns ± 2%     675ns ± 2%   -24.80%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~"(1|2|3|4|5|6|20|55)"-16        20.1ms ± 2%     0.0ms ± 2%   -99.99%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~".*"-16                         74.4ms ± 3%    81.3ms ± 3%    +9.29%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~"1.*"-16                        20.3ms ±27%    17.2ms ± 2%   -15.15%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~".*1"-16                        6.22ms ±10%    5.60ms ± 1%    -9.95%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~".+"-16                          146ms ± 4%     126ms ± 1%   -13.63%  (p=0.016 n=4+5)
PostingsForMatchers/Head/i=~""-16                           77.0ms ±23%    68.2ms ± 5%   -11.47%  (p=0.032 n=5+5)
PostingsForMatchers/Head/i!=""-16                           57.0ms ± 4%    56.9ms ± 6%      ~     (p=1.000 n=5+5)
PostingsForMatchers/Head/n="1",i=~".*",j="foo"-16           77.1ms ± 2%    79.2ms ± 4%      ~     (p=0.095 n=5+5)
PostingsForMatchers/Head/n="1",i=~".*",i!="2",j="foo"-16    77.2ms ± 3%    81.2ms ± 3%    +5.17%  (p=0.016 n=5+5)
PostingsForMatchers/Head/n="1",i!=""-16                     64.7ms ±34%    58.4ms ± 7%      ~     (p=0.421 n=5+5)

name                                                      old alloc/op   new alloc/op   delta
PostingsForMatchers/Head/n="1"-16                            80.0B ± 0%     80.0B ± 0%      ~     (all equal)
PostingsForMatchers/Head/n="1",j="foo"-16                     208B ± 0%      208B ± 0%      ~     (all equal)
PostingsForMatchers/Head/j="foo",n="1"-16                     208B ± 0%      208B ± 0%      ~     (all equal)
PostingsForMatchers/Head/n="1",j!="foo"-16                    240B ± 0%      240B ± 0%      ~     (all equal)
PostingsForMatchers/Head/i=~"1[0-9]",j=~"foo|bar"-16        1.61MB ± 0%    0.00MB ± 0%      ~     (p=0.079 n=4+5)
PostingsForMatchers/Head/j=~"foo|bar"-16                      360B ± 0%      232B ± 0%   -35.56%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~"(1|2|3|4|5|6|20|55)"-16        1.61MB ± 0%    0.00MB ± 0%   -99.97%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~".*"-16                         1.61MB ± 0%    1.61MB ± 0%      ~     (all equal)
PostingsForMatchers/Head/i=~"1.*"-16                        3.15MB ± 0%    3.15MB ± 0%    -0.00%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~".*1"-16                        1.61MB ± 0%    1.61MB ± 0%    -0.00%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~".+"-16                         17.3MB ± 0%    17.3MB ± 0%    -0.00%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~""-16                           17.3MB ± 0%    17.3MB ± 0%      ~     (p=0.238 n=5+5)
PostingsForMatchers/Head/i!=""-16                           17.3MB ± 0%    17.3MB ± 0%      ~     (all equal)
PostingsForMatchers/Head/n="1",i=~".*",j="foo"-16           1.61MB ± 0%    1.61MB ± 0%    -0.00%  (p=0.032 n=5+5)
PostingsForMatchers/Head/n="1",i=~".*",i!="2",j="foo"-16    1.61MB ± 0%    1.61MB ± 0%      ~     (p=0.651 n=5+4)
PostingsForMatchers/Head/n="1",i!=""-16                     17.3MB ± 0%    17.3MB ± 0%      ~     (all equal)

name                                                      old allocs/op  new allocs/op  delta
PostingsForMatchers/Head/n="1"-16                             4.00 ± 0%      4.00 ± 0%      ~     (all equal)
PostingsForMatchers/Head/n="1",j="foo"-16                     9.00 ± 0%      9.00 ± 0%      ~     (all equal)
PostingsForMatchers/Head/j="foo",n="1"-16                     9.00 ± 0%      9.00 ± 0%      ~     (all equal)
PostingsForMatchers/Head/n="1",j!="foo"-16                    10.0 ± 0%      10.0 ± 0%      ~     (all equal)
PostingsForMatchers/Head/i=~"1[0-9]",j=~"foo|bar"-16          19.0 ± 0%      23.0 ± 0%   +21.05%  (p=0.008 n=5+5)
PostingsForMatchers/Head/j=~"foo|bar"-16                      13.0 ± 0%       7.0 ± 0%   -46.15%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~"(1|2|3|4|5|6|20|55)"-16          4.00 ± 0%     13.00 ± 0%  +225.00%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~".*"-16                           7.00 ± 0%      7.00 ± 0%      ~     (all equal)
PostingsForMatchers/Head/i=~"1.*"-16                         11.1k ± 0%     11.1k ± 0%    -0.01%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~".*1"-16                          4.00 ± 0%      3.00 ± 0%   -25.00%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~".+"-16                           100k ± 0%      100k ± 0%    -0.00%  (p=0.008 n=5+5)
PostingsForMatchers/Head/i=~""-16                             100k ± 0%      100k ± 0%      ~     (all equal)
PostingsForMatchers/Head/i!=""-16                             100k ± 0%      100k ± 0%      ~     (all equal)
PostingsForMatchers/Head/n="1",i=~".*",j="foo"-16             12.0 ± 0%      12.0 ± 0%      ~     (all equal)
PostingsForMatchers/Head/n="1",i=~".*",i!="2",j="foo"-16      18.0 ± 0%      18.0 ± 0%      ~     (all equal)
PostingsForMatchers/Head/n="1",i!=""-16                       100k ± 0%      100k ± 0%      ~     (all equal)
```

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-06 14:16:14 +02:00
Ganesh Vernekar 9d4aa2604c
Merge pull request #18 from grafana/codesome/syncprom2 
Sync upstream
 2021-10-06 16:11:59 +05:30
Peter Štibraný b7b70066ae Address review feedback. 2021-10-06 11:04:37 +02:00
Ganesh Vernekar e37ff8f6b6
Merge remote-tracking branch 'upstream/main' into sync 2021-10-06 13:59:18 +05:30
Peter Štibraný d116268e59 Always include empty symbol in the symbol table. 2021-10-06 10:17:16 +02:00
Cyril Tovena 17a558336b
wip 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-06 09:13:40 +02:00
Cyril Tovena 0bb2d1954b
wip 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-06 09:12:45 +02:00
Julien Pivotto 5a4c3734b1
Merge pull request #9457 from prometheus/release-2.30-merge 
Merge release-2.30 branch into main
 2021-10-05 21:12:46 +02:00
Peter Štibraný 9a50267ea5 Delete symbols files. 2021-10-05 17:56:20 +02:00
Ganesh Vernekar 10d4cb6dc0
Merge remote-tracking branch 'upstream/main' into release-2.30-merge 2021-10-05 20:35:14 +05:30
Ganesh Vernekar f29caccc42
Cut v2.30.3 (#9446) 
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
 2021-10-05 20:31:30 +05:30
Cyril Tovena d217825af8
guards against too many matches. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-05 16:46:24 +02:00
Peter Štibraný 259e09fe5f When doing compaction with splitting, only use symbols from series that belong to given sharded block. 2021-10-05 16:06:38 +02:00
Cyril Tovena 639cd00721
Uses the variadic function. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-05 15:59:40 +02:00
Cyril Tovena 623c3ebeec
Remove redundant code and support charclass without base. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-05 15:16:21 +02:00
Cyril Tovena bd4d997bee
lint 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-05 14:00:18 +02:00
Cyril Tovena 7ea8609024
typo 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-05 13:54:55 +02:00
Cyril Tovena 453cc5de75
Add a comment for the charclass limit. 
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-05 13:52:12 +02:00
Cyril Tovena c751cfbfce
Improve findSetMatches to support concatenation. 
This improves how we find `SetMatches` for regexp. Notably it allows to support concatenation
such as `api_(v1|prom)_push` for which the resulting set matches are `api_v1_push` and `api_prom_push`.

I had to support characters classes too since the syntax may try to optimize alternates with them.

In the end the code is also more robust than the previous implementation relying on the stringyfied version of the regexp.

This could be upstreamed later.

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
 2021-10-05 13:43:41 +02:00