Commit graph

11115 commits

Author SHA1 Message Date
beorn7 fe50d6fc14 Update chunk layout documentation
Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-15 23:18:41 +02:00
beorn7 b8d953a5a0 scrape: Avoid creating a label map during conflict resolution
This also avoids the recursive function call. I think it is quite
readable. And much less code.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-15 21:56:48 +02:00
beorn7 ed33aea392 Avoid redundant varint decoding in chunk appender construction
Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-15 20:33:14 +02:00
Shirley Leu c890ea407f
Resolve conflicts between multiple exported label prefixes (#9479)
Resolve conflicts between multiple exported label prefixes

Signed-off-by: Shirley Leu <shirley.w.leu@gmail.com>
2021-10-15 20:31:03 +02:00
Howie 4e1dacf2d1
fix issue #9432(uses reference to loop iterator variable ) (#9483) 2021-10-15 14:24:55 -04:00
ziollek 55f9147b44
Add atan2 to scalar operators - issue #9485 (#9515)
* Add atan2 to scalar operators

Signed-off-by: Tomasz Ziolkowski <tomasz.ziolkowski@allegro.pl>
2021-10-15 16:03:11 +02:00
beorn7 d31bb75dc4 Use VarbitUint rather than VarbitInt to encode len(spans)
Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-15 15:27:32 +02:00
beorn7 3179215a59 Encode zero threshold first
This guaranees that the zero threshold is byte-aligned. Not sure if
that helps in any way, but at least it won't harm.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-14 14:55:21 +02:00
beorn7 c5522677bf Improve encoding of zero threshold
Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-14 14:47:26 +02:00
Ben Ye fdbc40a9ef
Expose NewChainSampleIterator func (#9475)
* expose NewChainSampleIterator func

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* add comment

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* update comments

Signed-off-by: Ben Ye <ben.ye@bytedance.com>
2021-10-14 14:49:00 +05:30
tclayton-newr e261eccb35
added doc for changed metric name in remote write (#9480)
* added doc for changed metric name in remote write

Signed-off-by: Tyler Clayton <tclayton@newrelic.com>
2021-10-14 00:12:54 +02:00
beorn7 7093b089f2 Use more varbit in histogram chunks
This adds bit buckets for larger numbers to varbit encoding and also
an unsigned version of varbit encoding.

Then, varbit encoding is used for all the histogram chunk data instead
of varint.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-13 20:03:35 +02:00
Björn Rabenstein 7309c20e7e
Merge pull request #9500 from codesome/resettests
Add unit test for counter reset header
2021-10-13 18:19:21 +02:00
Marco Pracucci fbd1b4fb84
Merge pull request #21 from grafana/reuse-expand-postings-method-in-new-postings-cloner
Use `ExpandPostings()` in `NewPostingsCloner()`
2021-10-13 17:01:24 +02:00
Ganesh Vernekar dcaf568279
Metadata -> Layout renaming
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-10-13 20:27:48 +05:30
Ganesh Vernekar 4e206c7c77
Fix reviews
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-10-13 20:23:31 +05:30
Oleg Zaytsev 14cb8d8696
Use ExpandPostings() in NewPostingsCloner()
We should reuse that method, not only because it saves code, but also
because that method is a good candidate to use a pool of slices some
day.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2021-10-13 15:26:36 +02:00
Dieter Plaetinck d5afe0a577
TSDB: Use a dedicated head chunk reference type (#9501)
* Use dedicated Ref type

Throughout the code base, there are reference types masked as
regular integers.  Let's use dedicated types.  They are
equivalent, but clearer semantically.
This also makes it trivial to find where they are used,
and from uses, find the centralized docs.

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* postpone some work until after possible return

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* clarify

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* rename feedback

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* skip header is up to caller

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>
2021-10-13 17:44:32 +05:30
Ganesh Vernekar 85e6686f84
Add unit test for counter reset header
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-10-13 15:57:42 +05:30
Björn Rabenstein 311673d62e
Save on slice allocations in histogramIterator (#9494)
Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-13 14:43:49 +05:30
Frederic Branczyk 054fc4ba19
Merge pull request #9467 from katsadim/endpointsliceovercapacity
discovery/kubernetes: Warn user in case of endpoint over-capacity
2021-10-13 10:57:23 +02:00
beorn7 468a89f1bf Merge branch 'sparsehistogram' of github.com:prometheus/prometheus into sparsehistogram 2021-10-12 15:30:48 +02:00
beorn7 94729d9936 Merge branch 'main' into sparsehistogram 2021-10-12 15:30:20 +02:00
Björn Rabenstein c450c01eb9
Remove obsolete TODOs about metadata (#9490)
Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-12 17:00:30 +05:30
Björn Rabenstein 9486be535b
Add deprecation notice for the pkg directory (#9477)
As per consensus expressed on the dev summit:
https://docs.google.com/document/d/11LC3wJcVk00l8w5P3oLQ-m3Y37iom6INAMEu2ZAGIIE/edit?pli=1#heading=h.80qrixk0sjgv

Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-12 10:54:12 +02:00
Julien Pivotto e245edf5a4
Fix Levi's name in MAINTAINERS.md (#9491)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-10-12 02:05:22 +02:00
Björn Rabenstein 354d8d2ecf
Merge pull request #9488 from prometheus/beorn7/release
Volunteer @roidelapluie and @juliusv as upcoming release shepherds
2021-10-11 19:14:44 +02:00
Julien Pivotto 81cb67eb95
Add Levi as maintainer (#9487)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-10-11 17:46:59 +02:00
beorn7 810c1081cf Volunteer @roidelapluie and @juliusv as upcoming release shepherds
Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-11 17:14:32 +02:00
Marco Pracucci 8bc1b41795
Merge pull request #20 from grafana/test-approxBytesPerEntry-is-correct
Test that approxBytesPerEntry const is correct
2021-10-11 15:42:38 +02:00
Oleg Zaytsev 3dd18a1ece
Test that approxBytesPerEntry const is correct
The comment on the constant mentions where it comes from, so we can
actually check its value in that test.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2021-10-11 15:15:56 +02:00
Cyril Tovena 62935a1241
Merge pull request #19 from grafana/regexpopti
Optimize most common regexp
2021-10-11 14:15:51 +02:00
Björn Rabenstein 12486b1250
Merge pull request #9481 from prometheus/beorn7/cleanup
Style cleanup of all the changes in sparsehistogram so far
2021-10-11 14:01:36 +02:00
Cyril Tovena 01187fbce2
Review feedback
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
2021-10-11 13:59:24 +02:00
Cyril Tovena c8dfba9fa4
Add better comments
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
2021-10-11 13:56:22 +02:00
beorn7 7a8bb8222c Style cleanup of all the changes in sparsehistogram so far
A lot of this code was hacked together, literally during a
hackathon. This commit intends not to change the code substantially,
but just make the code obey the usual style practices.

A (possibly incomplete) list of areas:

* Generally address linter warnings.

* The `pgk` directory is deprecated as per dev-summit. No new packages should
  be added to it. I moved the new `pkg/histogram` package to `model`
  anticipating what's proposed in #9478.

* Make the naming of the Sparse Histogram more consistent. Including
  abbreviations, there were just too many names for it: SparseHistogram,
  Histogram, Histo, hist, his, shs, h. The idea is to call it "Histogram" in
  general. Only add "Sparse" if it is needed to avoid confusion with
  conventional Histograms (which is rare because the TSDB really has no notion
  of conventional Histograms). Use abbreviations only in local scope, and then
  really abbreviate (not just removing three out of seven letters like in
  "Histo"). This is in the spirit of
  https://github.com/golang/go/wiki/CodeReviewComments#variable-names

* Several other minor name changes.

* A lot of formatting of doc comments. For one, following
  https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences
  , but also layout question, anticipating how things will look like
  when rendered by `godoc` (even where `godoc` doesn't render them
  right now because they are for unexported types or not a doc comment
  at all but just a normal code comment - consistency is queen!).

* Re-enabled `TestQueryLog` and `TestEndopints` (they pass now,
  leaving them disabled was presumably an oversight).

* Bucket iterator for histogram.Histogram is now created with a
  method.

* HistogramChunk.iterator now allows iterator recycling. (I think
  @dieterbe only commented it out because he was confused by the
  question in the comment.)

* HistogramAppender.Append panics now because we decided to treat
  staleness marker differently.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-10-11 13:02:03 +02:00
Cyril Tovena d41e470693
Update pkg/labels/regexp.go
Co-authored-by: Marco Pracucci <marco@pracucci.com>
2021-10-11 11:13:53 +02:00
Cyril Tovena 37416bf907
Update pkg/labels/regexp.go
Co-authored-by: Marco Pracucci <marco@pracucci.com>
2021-10-11 11:09:55 +02:00
Cyril Tovena 5c5b38712f
Add more comments.
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
2021-10-11 10:10:51 +02:00
Cyril Tovena 9c33f392e6
Better benchmarks values.
```
❯ benchcmp  before.txt after.txt
benchmark                                                           old ns/op     new ns/op     delta
BenchmarkNewFastRegexMatcher/(foo|bar)-16                           2487          2520          +1.33%
BenchmarkNewFastRegexMatcher/foo.*-16                               8856          4531          -48.84%
BenchmarkNewFastRegexMatcher/.*foo-16                               25195         6389          -74.64%
BenchmarkNewFastRegexMatcher/^.*foo$-16                             25626         6253          -75.60%
BenchmarkNewFastRegexMatcher/^.+foo$-16                             25429         6248          -75.43%
BenchmarkNewFastRegexMatcher/.*-16                                  429707        8439          -98.04%
BenchmarkNewFastRegexMatcher/.+-16                                  380165        8503          -97.76%
BenchmarkNewFastRegexMatcher/foo.+-16                               8180          4586          -43.94%
BenchmarkNewFastRegexMatcher/.+foo-16                               25214         6255          -75.19%
BenchmarkNewFastRegexMatcher/foo_.+-16                              8116          4334          -46.60%
BenchmarkNewFastRegexMatcher/foo_.*-16                              8354          4287          -48.68%
BenchmarkNewFastRegexMatcher/.*foo.*-16                             206076        19227         -90.67%
BenchmarkNewFastRegexMatcher/.+foo.+-16                             208434        18793         -90.98%
BenchmarkNewFastRegexMatcher/#00-16                                 33045         3936          -88.09%
BenchmarkNewFastRegexMatcher/(?s:.*)-16                             403806        4208          -98.96%
BenchmarkNewFastRegexMatcher/(?s:.+)-16                             418177        4150          -99.01%
BenchmarkNewFastRegexMatcher/(?s:^.*foo$)-16                        24452         5661          -76.85%
BenchmarkNewFastRegexMatcher/^(?i:foo|oo)|(bar)$-16                 48087         23183         -51.79%
BenchmarkNewFastRegexMatcher/((.*)(bar|b|buzz)(.+)|foo)$-16         645430        47193         -92.69%
BenchmarkNewFastRegexMatcher/^$-16                                  37244         3912          -89.50%
BenchmarkNewFastRegexMatcher/(prometheus|api_prom)_api_v1_.+-16     17205         10006         -41.84%
BenchmarkNewFastRegexMatcher/10\.0\.(1|2)\.+-16                     6776          7011          +3.47%
BenchmarkNewFastRegexMatcher/10\.0\.(1|2).+-16                      14792         4674          -68.40%
BenchmarkNewFastRegexMatcher/((fo(bar))|.+foo)-16                   497007        17410         -96.50%
```

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
2021-10-11 09:45:02 +02:00
akatsadimas ee77a6212f discovery/kubernetes: issue overcapacity warning for endpoint rather than endpointslice
Signed-off-by: akatsadimas <nkatsadim@gmail.com>
2021-10-08 23:17:04 +03:00
Cyril Tovena 715274ec9d
Reintroduce tests.
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
2021-10-08 10:10:18 +02:00
beorn7 fd5ea4e0b5 Merge branch 'main' into sparsehistogram 2021-10-07 23:16:42 +02:00
Björn Rabenstein 96f0683318
Merge pull request #9439 from codesome/counterresetmeta
Add info about counter resets in chunk meta
2021-10-07 21:42:55 +02:00
Cyril Tovena 7dab60e56d
More optimization.
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
2021-10-07 17:15:20 +02:00
Ganesh Vernekar 5d4dc7e413
Convert the header into an enum
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-10-07 19:53:24 +05:30
Christian Simon 50c1060328
Implement PostingsCloner and implement promise/future pattern for PostingsForMatchers (#14)
* Fix tests not closing head/block properly

Close Head properly in TestWalRepair_DecodingError
Close Block properly in  TestReadIndexFormatV1

Co-authored-by: Christian Simon <simon@swine.de>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Implement PostingsCloner

PostingsCloner allows obtaining independend clones of a Postings, that
can be used for subsequent calls.

Co-authored-by: Christian Simon <simon@swine.de>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* IndexReader.PostingsForMatchers() and implement Cache

IndexReader now offers the PostingsForMatchers functionality, which is
provided by PostingsForMatchersCache.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>

Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>
2021-10-07 15:16:09 +02:00
Cyril Tovena 852b553c00
Fixes the code and improve tests.
Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>
2021-10-07 15:10:26 +02:00
Cyril Tovena 9173cade01
Merge pull request #15 from grafana/improvesetmatches
Improve `findSetMatches `to support concatenation.
2021-10-07 14:48:53 +02:00
Marco Pracucci 23ac1d73ef
Fixed timeout increase
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2021-10-07 14:35:55 +02:00