mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-27 14:39:40 -08:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity
1075 commits 243 branches 444 tags 649 MiB
Commit graph

1075 commits

This branch
This branch
All branches
Author SHA1 Message Date
Tom Wilkie 77d5a7d47a
LiveReader can get into an infinite loop on corrupt WALs. (#524) 
Make WAL live tailer return EOF when the there is a half-written record at the end of the file.

Previously, this would cause an infinite loop as we ignored EOFs when filling the buffer.  We now differentiate between EOFs that read >0 bytes, and EOFs that didn't.

Add some more unit tests for tailing a corrupt WAL, and unify interfaces Reader and LiveReader for the purposes of testing.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
 2019-02-19 14:33:57 +00:00
Krasi Georgiev 8913617a9e
update makefile.common and run make all be default. (#529) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-19 13:55:01 +02:00
Tom Wilkie bc3b0bd429
Test to corrupt segments mid-WAL, repair and check we can read the correct number of records. (#528) 
Test to corrupt segments mid-WAL, repair and check we can read the correct number of records.

Make segmentBufReader pad short segments with zeros, and only advance curr segment index after fully reading segment.
 2019-02-18 19:05:07 +00:00
Ganesh Vernekar c59ed492b2 Vertical query merging and compaction (#370) 
* Vertical series iterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Select overlapped blocks first in compactor Plan()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added vertical compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Code cleanup and comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add benchmark for compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Perform vertical compaction only when blocks are overlapping.

Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for vertical compaction

* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for query iterator and seek for non overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Vertical query merge only for overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Simplify logging in Compact(...)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Calculate overlapping inside populateBlock

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* MinTime and MaxTime for BlockReader.

Using this to find overlapping blocks in populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Sort blocks w.r.t. MinTime in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping in LeveledCompactor.write() instead of returning bool

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping inside LeveledCompactor.populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor createBlock to take optional []Series

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* review1

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* Updated CHANGELOG and minor nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor iterator and seek benchmarks for Querier.

Also has as overlapping blocks.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Additional test case

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Split genSeries into genSeries and populateSeries

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Check error in benchmark

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Warn about overlapping blocks in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-14 14:29:41 +01:00
Callum Styan 89ee5aaed4 clarify which segments are deleted when we find a corrupted segment (#522) 
Signed-off-by: Callum Styan <callumstyan@gmail.com>
 2019-02-14 05:44:19 +02:00
Bryan Boreham 74093f0508 Remove pointer indirection on chunk bstream (#499) 
It isn't necessary since the member is never changed after
initialization.

Signed-off-by: Bryan Boreham <bryan@weave.works>
 2019-02-13 23:41:12 +01:00
Alec 109252f3aa Update encoding_helpers.go (len of be64 should be 8) (#521) 2019-02-13 11:04:21 +02:00
Krasi Georgiev 9f28ffa6f4
Merge pull request #465 from krasi-georgiev/shutdown-during-compaction 
use context to cancel compactions
 2019-02-12 11:25:40 +02:00
Krasi Georgiev bf79c767f0 new line 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 11:08:09 +02:00
Krasi Georgiev beee5c58f3 Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 10:56:45 +02:00
Krasi Georgiev 6fce018def
Merge pull request #512 from krasi-georgiev/delete-compact-block-on-reload-error 
Delete compact block on reload error
 2019-02-12 10:50:21 +02:00
Krasi Georgiev bf2239079d refactor multi errors 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 12:28:46 +02:00
Krasi Georgiev c3a5c1d891 refactor error handling 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 12:22:11 +02:00
Krasi Georgiev 07df4fd383 nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 11:25:57 +02:00
Krasi Georgiev 0f8f5027ef remove nested for if 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 18:09:23 +02:00
Krasi Georgiev d48606827c simplify closers 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 13:35:32 +02:00
Krasi Georgiev e138c7ed7e Merge remote-tracking branch 'upstream/master' into delete-compact-block-on-reload-error 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 13:26:28 +02:00
Krasi Georgiev da9da9fbee fix the sleep logic 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 12:39:25 +02:00
Krasi Georgiev 0b72f9af4c
Merge pull request #270 from codesome/master 
Head: don't create stones, delete samples directly
 2019-02-08 12:35:01 +02:00
Krasi Georgiev 1e91d619de
Merge pull request #493 from simonpasquier/update-makefile-common 
fix static check errors and removed a lot of unused code and vars.
updated Makefile.common
 2019-02-08 11:47:06 +02:00
Krasi Georgiev 457534d5c4 simplify nesting. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 11:36:30 +02:00
Ganesh Vernekar 5481549324
Merge remote-tracking branch 'upstream/master' 
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-07 22:15:06 +05:30
Simon Pasquier 272fd0eabf Update Makefile.common 
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
 2019-02-07 15:06:10 +01:00
Simon Pasquier 95334f13c5 Merge branch 'master' into update-makefile-common 
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
 2019-02-07 12:10:22 +01:00
Krasi Georgiev 2ae0620205 rename some vars and use Gauge instead of Counter for metrics 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-07 10:09:42 +02:00
Krasi Georgiev 776769377e fix merr logic. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-06 16:59:28 +02:00
Krasi Georgiev 08e7bc8ee8 always remove tmp 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-06 16:09:42 +02:00
Krasi Georgiev 45acaadd81 review changes 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-06 14:07:35 +02:00
Krasi Georgiev 77cc2ea210
Update index.md(format of series entry) (#515) 
* Update index.md
* more clear format representation
* move dots to the bottom
* update #labels count, #chunks count
* use correct variable types.

Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 12:13:38 +03:00
naivewong 9b227d27e7 update labels count, chunks count 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 10:00:18 +08:00
naivewong 8e7e2041d3 update #labels count, #chunks count 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 00:48:25 +08:00
naivewong 42ee59689b move dots to the bottom 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 00:34:15 +08:00
naivewong 0649dfddf0 more clear format representation 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 00:05:30 +08:00
naivewong 1f723a8eb5 update #labels and #chunks 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-05 23:20:43 +08:00
Alec 8e589474c6 Update index.md 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-05 10:34:25 +08:00
Alec 2b6bc9fb32 Update index.md 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-05 10:34:25 +08:00
Alec 1bcda9d23f Update tombstones.md (format of tombstone) (#511) 2019-02-04 12:30:47 +03:00
Krasi Georgiev ce4a2083fb nit 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-04 11:15:08 +02:00
Krasi Georgiev 752ab86e4e change the test block series for more stable tests 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-04 11:14:39 +02:00
Krasi Georgiev 00d5e19baf Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 13:47:50 +02:00
Krasi Georgiev 6c34eb8b63 nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 12:17:15 +02:00
Krasi Georgiev 315de4c782 fix windows tests 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 11:40:40 +02:00
Krasi Georgiev 1603222bbc small refactor of openTestDB to handle errors properly. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 11:40:12 +02:00
Krasi Georgiev 7245c6dc33 Merge remote-tracking branch 'upstream/master' into delete-compact-block-on-reload-error 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 11:39:19 +02:00
Krasi Georgiev ee99718ff6
rename chunk reader vars to make it easier to follow. (#508) 
* rename chunk reader vars to make it easyer to understand.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-29 20:46:12 +03:00
radek_lesniewski eb5034d5b0 Additional logging in compact.go - logged time needed for writing blocks (#505) 
* Additional logging in compact.go - logged time needed for writing blocks to disk

Signed-off-by: Radoslaw Lesniewski <Radoslaw.Lesniewski@sabre.com>

* Additional logging in compact.go - code formatted

Signed-off-by: Radoslaw Lesniewski <Radoslaw.Lesniewski@sabre.com>
 2019-01-29 16:53:53 +05:30
Krasi Georgiev 3ec08eac50 use camelcase for rangeToTriggerCompaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-29 10:32:59 +02:00
Krasi Georgiev 53c18e7a41 use a global indexFilename constant 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-29 10:32:32 +02:00
Krasi Georgiev b521559c3b do a proper cleanup for a failed reload after a compaction 
a failed reload immediately after a compaction should delete the
resulting block to avoid creating blocks with the same time range.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-29 10:31:54 +02:00
Ganesh Vernekar a29ab50a20
Merge pull request #510 from yeya24/patch/fixtypo 
fix two typos in db_test
 2019-01-29 11:51:03 +05:30