mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-25 05:34:05 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
973 commits 243 branches 444 tags 649 MiB
Commit graph

40 commits

Author SHA1 Message Date
Ganesh Vernekar c59ed492b2 Vertical query merging and compaction (#370) 
* Vertical series iterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Select overlapped blocks first in compactor Plan()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added vertical compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Code cleanup and comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add benchmark for compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Perform vertical compaction only when blocks are overlapping.

Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for vertical compaction

* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for query iterator and seek for non overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Vertical query merge only for overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Simplify logging in Compact(...)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Calculate overlapping inside populateBlock

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* MinTime and MaxTime for BlockReader.

Using this to find overlapping blocks in populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Sort blocks w.r.t. MinTime in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping in LeveledCompactor.write() instead of returning bool

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping inside LeveledCompactor.populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor createBlock to take optional []Series

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* review1

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* Updated CHANGELOG and minor nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor iterator and seek benchmarks for Querier.

Also has as overlapping blocks.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Additional test case

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Split genSeries into genSeries and populateSeries

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Check error in benchmark

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Warn about overlapping blocks in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-14 14:29:41 +01:00
Krasi Georgiev bf79c767f0 new line 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 11:08:09 +02:00
Krasi Georgiev beee5c58f3 Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 10:56:45 +02:00
Krasi Georgiev 07df4fd383 nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 11:25:57 +02:00
Krasi Georgiev 0f8f5027ef remove nested for if 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 18:09:23 +02:00
Krasi Georgiev da9da9fbee fix the sleep logic 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 12:39:25 +02:00
Krasi Georgiev 457534d5c4 simplify nesting. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 11:36:30 +02:00
Krasi Georgiev 752ab86e4e change the test block series for more stable tests 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-04 11:14:39 +02:00
Krasi Georgiev 6c34eb8b63 nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 12:17:15 +02:00
Krasi Georgiev 1603222bbc small refactor of openTestDB to handle errors properly. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 11:40:12 +02:00
Krasi Georgiev b521559c3b do a proper cleanup for a failed reload after a compaction 
a failed reload immediately after a compaction should delete the
resulting block to avoid creating blocks with the same time range.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-29 10:31:54 +02:00
Krasi Georgiev fa1c00f9e2 no backgorund 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-24 14:38:45 +02:00
Krasi Georgiev 1b0d85bbf2 use a db copy instead of creating it again. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-24 14:15:32 +02:00
Krasi Georgiev 8ffd705346 fix the misleading log during compaction cancelation 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-24 13:33:12 +02:00
Krasi Georgiev 9638c13ec8 test compares normal vs canceled compaction times 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-24 12:48:56 +02:00
Krasi Georgiev 9874377ead nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-18 19:22:45 +02:00
Krasi Georgiev 921f82cfc1 WIP 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-18 18:58:17 +02:00
Krasi Georgiev 0b31b8af88 Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-18 16:58:18 +02:00
Krasi Georgiev a2779cc901
fix flaky tests: TestDisableAutoCompactions,TestBlockRanges (#472) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-12 14:49:03 +03:00
Krasi Georgiev db127a60e0 changelog 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-10 12:48:30 +02:00
Krasi Georgiev fced260a24 test 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-10 12:48:30 +02:00
Krasi Georgiev bd5ccee5c1 use context to cancel compactions 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-10 12:48:30 +02:00
Krasi Georgiev bac9cbed2e
no overlapping on compaction when an existing block is not within default boundaries. (#461) 
closes https://github.com/prometheus/prometheus/issues/4643

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-04 13:30:49 +03:00
Krasi Georgiev b75d702ceb
fix flaky compaction test (#458) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-11-21 00:33:14 +02:00
Krasi Georgiev 7f00217d77
Allow manual compaction for tests when compaction is disabled globally. (#412) 
for tests we need to control when a compaction happens so with this
change automated compaction can be disabled, but allow to run it
manually it tests.

fixes failing tests in : https://github.com/prometheus/tsdb/pull/374

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-11-20 12:34:26 +02:00
Bartek Płotka 047b1b1357 compact: Verify for chunks outside of compacted time range. Added unit test for populateBlocs. (#349) 
* compact: Verify for chunks outside of compacted time range. 
 Unit test for populateBlocs.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Co-authored-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-10-12 12:45:19 +03:00
Bartek Plotka 2aceac0a12 Fixed some comments. 
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2018-03-13 15:30:00 +00:00
Bartek Plotka 328c0ff5b0 Simplified the flow and tests. 
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2018-03-13 14:12:32 +00:00
Bartek Plotka 483da43660 compact: Exclude last block with shortest range instead of newest one by ULID. 
Fixes https://github.com/prometheus/prometheus/issues/3943 issue.

Added tests.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2018-03-13 12:30:27 +00:00
Krasi Georgiev eb6cdbcb5b some test funcitons didn't cleanup after themselves. 2018-02-23 18:45:59 +02:00
Callum Styan ca4e817372 move test utils to it's own package so we can use it in tests for other packages in tsdb repo 2017-12-06 17:06:14 -08:00
Callum Styan fa26e2b6c3 use test utils in compact_test 2017-12-06 16:30:24 -08:00
Goutham Veeramachaneni 1fc94a02d1
Don't retry failed compactions. 
Fixes prometheus/prometheus#3487

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-11-23 15:16:20 +05:30
Fabian Reinartz 4cc37eecab Refactor and add tests for compactor 2017-09-01 11:50:58 +02:00
Goutham Veeramachaneni 178f840295 Fix panic caused by 0 division 
Introduced in #108

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-08-18 07:45:29 +02:00
Fabian Reinartz 905af27cf9 Refactor compactor 2017-08-09 11:10:29 +02:00
Fabian Reinartz 3951d8cc29 Fix selection of compactable blocks 2017-08-04 12:02:34 +02:00
Fabian Reinartz 5d7b5994d6 Fix compaction range selection 2017-07-13 16:13:59 +02:00
Goutham Veeramachaneni 9a2ab732b5 Choose ranges by retention and min-block-duration 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 17:30:34 +02:00
Goutham Veeramachaneni ce23adb579 Initial compaction implementation for block-ranges 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-07 16:32:18 +02:00