Commit graph

1029 commits

Author SHA1 Message Date
Brian Brazil 6ac81cc7a9
Correctly handle empty labels. (#594)
Currently a time series with empty labels is not treated the same
as one with missing labels. Currently this can only come from
ALERTS&ALERT_FOR_STATE so it's unlikely anyone has actually hit it.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-05-07 11:00:16 +01:00
Goutham Veeramachaneni 2ae028114c
Add FromData function (again)! (#600)
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2019-05-06 19:17:26 +05:30
PrometheusBot 4da65c46c9 makefile: update Makefile.common with newer version (#597)
Signed-off-by: prombot <prometheus-team@googlegroups.com>
2019-05-05 19:17:35 +03:00
zhulongcheng 3ccab17f5d docs: add label offset table and postings offset table (#593)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-05-02 23:23:58 +03:00
Krasi Georgiev 5512826f13
make Close methods for the querier safe to call more than once. (#581)
* make Close methods for the querier safe to call more than once.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-04-30 10:17:07 +03:00
zhulongcheng 19d402d154 cleanup tmp files (#570)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-04-26 11:27:36 +03:00
Krasi Georgiev 288f67efbf
fix some unchecked errors and remove unused vars. (#592)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-04-25 13:07:04 +03:00
PrometheusBot a10c001689 Synchronize Makefile.common from prometheus/prometheus (#591)
* makefile: update Makefile.common with newer version

Signed-off-by: prombot <prometheus-team@googlegroups.com>

* Add .golangci.yml

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-25 12:29:09 +03:00
Brian Brazil c20450564c
Release 0.7.1 (#587)
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-24 13:20:45 +01:00
Brian Brazil f9bc0817a1
Avoid allocation in mergedPostings.Seek. (#586)
See
https://github.com/prometheus/prometheus/issues/5424#issuecomment-485199651

benchmark                                                            old
ns/op      new ns/op       delta
BenchmarkHeadPostingForMatchers/n="1"-4
5266           5267            +0.02%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4
6469           7801            +20.59%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4
4984           4963            -0.42%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4
7137           7527            +5.46%
BenchmarkHeadPostingForMatchers/i=~".*"-4
8055830839     8117651746      +0.77%
BenchmarkHeadPostingForMatchers/i=~".+"-4
9369298293     9742710251      +3.99%
BenchmarkHeadPostingForMatchers/i=~""-4
2363120708     2507789029      +6.12%
BenchmarkHeadPostingForMatchers/i!=""-4
9387069195     11740628557     +25.07%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4
109668312      107644136       -1.85%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4
106022679      105759760       -0.25%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4
330201810      266421024       -19.32%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4
355627801      292774913       -17.67%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4
108132305      106697690       -1.33%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4
358836972      362579998       +1.04%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4
555879392      575858378       +3.59%

benchmark                                                            old
allocs     new allocs     delta
BenchmarkHeadPostingForMatchers/n="1"-4                              8
8              +0.00%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4                      11
11             +0.00%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4                      11
11             +0.00%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4                     17
17             +0.00%
BenchmarkHeadPostingForMatchers/i=~".*"-4                            58
58             +0.00%
BenchmarkHeadPostingForMatchers/i=~".+"-4
100115         100115         +0.00%
BenchmarkHeadPostingForMatchers/i=~""-4
100125         100125         +0.00%
BenchmarkHeadPostingForMatchers/i!=""-4
100112         100112         +0.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4              19
19             +0.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4       22
22             +0.00%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4
100109         100079         -0.03%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4
100110         100079         -0.03%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4
11176          11167          -0.08%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4
100112         100082         -0.03%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4
111307         111277         -0.03%

benchmark                                                            old
bytes     new bytes     delta
BenchmarkHeadPostingForMatchers/n="1"-4
1360          1360          +0.00%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4
1392          1392          +0.00%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4
1392          1392          +0.00%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4
1520          1520          +0.00%
BenchmarkHeadPostingForMatchers/i=~".*"-4
258254040     258254040     +0.00%
BenchmarkHeadPostingForMatchers/i=~".+"-4
281554280     281554280     +0.00%
BenchmarkHeadPostingForMatchers/i=~""-4
241554888     241554888     +0.00%
BenchmarkHeadPostingForMatchers/i!=""-4
281553664     281553664     +0.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4
1607277       1607277       +0.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4
1607389       1607389       +0.00%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4
73076560      24907600      -65.92%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4
73076765      24907723      -65.92%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4
5416925       3794909       -29.94%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4
73076779      24907819      -65.92%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4
99874860      51705900      -48.23%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-24 13:12:58 +01:00
naivewong 7a12d4588a Avoid potential collision of createIdxChkReaders in querier_test.go (#585)
* Make chink ref monotonically increasing number in the `createIdxChkReaders` test func.
Signed-off-by: naivewong <867245430@qq.com>
2019-04-18 16:11:39 +03:00
PrometheusBot e2c6eb2c60 Synchronize Makefile.common from prometheus/prometheus (#584)
Propagating changes from master Makefile.common located in prometheus/prometheus.
2019-04-18 14:26:25 +03:00
Krasi Georgiev 4b3a5ac5d3
release 0.7.0 (#580)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-04-10 14:58:20 +03:00
Brian Brazil dfed85e4a4
Keep series that are still in WAL in checkpoints (#577)
If all the samples are deleted for a series,
we should still keep the series in the WAL as
anything else reading the WAL will still care
about it in order to understand the samples.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-09 14:16:24 +01:00
Brian Brazil 259847a6b1
Be smarter in how we look at matchers. (#572)
* Add unittests for PostingsForMatcher.

* Selector methods are all stateless, don't need a reference.

* Be smarter in how we look at matchers.

Look at all matchers to see if a label can be empty.

Optimise Not handling, so i!="2" is a simple lookup
rather than an inverse postings list.

All all the Withouts together, rather than
having to subtract each from all postings.

Change the pre-expand the postings logic to always do it before doing a
Without only. Don't do that if it's already a list.

The initial goal here was that the oft-seen pattern
i=~"something.+",i!="foo",i!="bar" becomes more efficient.

benchmark                                                            old ns/op     new ns/op     delta
BenchmarkHeadPostingForMatchers/n="1"-4                              5888          6160          +4.62%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4                      7190          6640          -7.65%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4                      6038          5923          -1.90%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4                     6030884       4850525       -19.57%
BenchmarkHeadPostingForMatchers/i=~".*"-4                            887377940     230329137     -74.04%
BenchmarkHeadPostingForMatchers/i=~".+"-4                            490316101     319931758     -34.75%
BenchmarkHeadPostingForMatchers/i=~""-4                              594961991     130279313     -78.10%
BenchmarkHeadPostingForMatchers/i!=""-4                              537542388     318751015     -40.70%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4              10460243      8565195       -18.12%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4       44964267      8561546       -80.96%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4                42244885      29137737      -31.03%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4              35285834      32774584      -7.12%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4             8951047       8379024       -6.39%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4       63813335      30672688      -51.93%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4     45381112      44924397      -1.01%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-09 11:59:45 +01:00
naivewong 7ab060c864 selectOverlappingDirs selects wrong dirs where there are 2 disjoint sets of overlapping dirs
Signed-off-by: naivewong <867245430@qq.com>
2019-04-08 15:27:06 +03:00
zhulongcheng 8b33ee9e2b force persisting the tombstone file (#578)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-04-08 15:06:40 +03:00
Pavel Borzenkov 520b1d858f index: add a test to trigger fd leak on corrupted index (#576)
The test is designed to fail for windows when the function leaves open files.
2019-04-03 11:34:04 +03:00
Krasi Georgiev 8eeb70fee1
remove Fsync workaround for macos. (#574)
since golang 1.12 no special handling is required for file.Sync()

@pborzenkov thanks for the pointer.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-04-03 11:16:54 +03:00
Pavel Borzenkov 4f204dcbc1 chunks: close all opened MmapFiles on NewDirReader error path (#575)
Signed-off-by: Pavel Borzenkov <pavel.borzenkov@gmail.com>
2019-04-02 15:16:29 +03:00
Krasi Georgiev e46ec89de6
use fsync to force persisting the meta file. (#573)
* use fsync to force persisting the meta file.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-04-01 16:23:45 +03:00
zhulongcheng aed16621c0 Add Head.compactable method (#542)
* Add Head.compactable method

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-04-01 11:19:06 +03:00
Vladimir Masarik fd188be052 Fixed default benchmark test path (#571)
Signed-off-by: Vladimir Masarik <vmasarik@redhat.com>
2019-03-27 16:33:59 +02:00
Vishnunarayan K I 7757fe6f21 Added ability to create db with NopWal (#519)
Signed-off-by: Vishnunarayan K I <appukuttancr@gmail.com>
2019-03-26 01:38:12 +02:00
zhulongcheng db9177de0c docs: add multi chunks layout in chunks readme file (#567) 2019-03-25 12:33:38 +02:00
zhulongcheng e11e01f68d Fix exported function comments (#566)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-25 10:17:28 +02:00
zhulongcheng 4d03c70800 Verify chunk format version (#544)
Verify chunk format version.
2019-03-24 22:33:08 +02:00
Bartek Płotka 3ab5f4e579 index: reduce empty postings trees (#509)
Improved Merge when all is empty.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2019-03-21 18:23:00 +02:00
naivewong 1c392d0057 Update wal.md that the first wal entry includes only base id and timestamp without a value (#562)
Signed-off-by: naivewong <867245430@qq.com>
2019-03-21 11:21:50 +02:00
zhulongcheng 837ae9aaa0 Update comment for ErrOutOfOrderSample (#563)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-21 10:53:39 +02:00
zhulongcheng 62cfe4446f Make Head.symbols map with size hint (#552)
To reduce the number of times the map is resized

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-20 10:43:07 +02:00
zhulongcheng 921da4dced Remove unused statement from index test (#558)
Remove unused `i++` statement from index test.
2019-03-19 15:41:40 +02:00
Krasi Georgiev c3ffdf1a99
Test createBlock and check all os.RemoveAll in the tests for errors. (#549)
Testing that createBlock creates blocks that can be opened.

and checking the os.RemoveAll for errors will catch errors for un-closed files under windows.

Many missing .Close() calls were added for fixing failing os.RemoveAll

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-03-19 15:31:57 +02:00
naivewong dd0d3c6f02 Several small fixes (#550)
`if ds[i].meta.MinTime < t0 || ds[i].meta.MaxTime > t0+tr `, ds[i].meta.MinTime is always larger or equal to t0, so no need for this check.
`ulid.Parse` only checks if the length is 26. So changed to  using `ulid.ParseStrict` to also check the validity of ulid.
2019-03-18 16:14:10 +02:00
zhulongcheng 99703b3269 README: move the build badge to the badge list (#555)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-18 14:44:58 +02:00
zhulongcheng 95648b33c4 Fix a typo in head.go (#553)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-18 10:37:20 +02:00
zhulongcheng 8feb67e630 Update Appender.AddFast method comment (#551)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-17 07:42:18 +02:00
zhulongcheng b7cac504c9 Remove unused NewReader function (#547)
The `NewReader` function is not used anywhere.
2019-03-14 15:05:40 +02:00
Simon Pasquier 84e51f7222 Update Makefile.common (#548)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-03-13 10:40:17 +02:00
zhulongcheng 719b57db44 Remove obsolete FromData function (#540)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-06 11:12:28 +02:00
Ganesh Vernekar 8921f18eb3
Merge pull request #543 from codesome/v0.6.1
*:cut 0.6.1
2019-03-05 21:36:06 +05:30
Ganesh Vernekar fcfe312d99
Update CHANGELOG for v0.6.1
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-03-05 19:21:46 +05:30
Alec 0861a9b14f [bugfix] update 'last' variable in chunks.MergeOverlappingChunks() (#539)
* update

Signed-off-by: naivewong <867245430@qq.com>

* update comment

Signed-off-by: naivewong <867245430@qq.com>

* Update failing test.

Signed-off-by: naivewong <867245430@qq.com>

* Update comment

Signed-off-by: naivewong <867245430@qq.com>

* Add changelog entry.

Signed-off-by: naivewong <867245430@qq.com>

* Update CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-03-05 19:18:55 +05:30
Simon Pasquier 08af7bbf90 Add Go 1.12 to Travis CI (#538)
Also update to the latest version of Makefile.common.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-03-04 21:44:40 +02:00
Krasi Georgiev ab8476dd06
Release 0.6.0 (#537) 2019-03-02 15:54:49 +02:00
Krasi Georgiev e78fedf054
release 0.5.0 (#536)
* release 0.5.0

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-03-01 22:57:53 +02:00
Brian Brazil 62b652fbd0
Improve Merge performance (#531)
Use a heap for Next for merges, and
pre-compute if there's many postings on the
unset path.

Add posting lookup benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-02-28 17:23:55 +00:00
Krasi Georgiev df06f9ebc2
update the CHANGELOG with some missing changes. (#535)
missed to add these with some merged PRs

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-27 12:55:35 +02:00
Ganesh Vernekar 158c3074cd
Merge pull request #534 from codesome/optional-vertical-blocks
Make vertical compaction and query merge optional
2019-02-26 13:50:05 -08:00
Ganesh Vernekar 28c73f531f
Make vertical compaction and query merge optional
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-02-26 11:50:50 -08:00