mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-09-20 15:57:31 -07:00
Code Issues Actions 7 Packages Projects Releases Wiki Activity
983 commits 210 branches 425 tags 635 MiB
Commit graph

983 commits

This branch
This branch
All branches
Author SHA1 Message Date
Brian Brazil 62b652fbd0
Improve Merge performance (#531) 
Use a heap for Next for merges, and
pre-compute if there's many postings on the
unset path.

Add posting lookup benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2019-02-28 17:23:55 +00:00
Krasi Georgiev df06f9ebc2
update the CHANGELOG with some missing changes. (#535) 
missed to add these with some merged PRs

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-27 12:55:35 +02:00
Ganesh Vernekar 158c3074cd
Merge pull request #534 from codesome/optional-vertical-blocks 
Make vertical compaction and query merge optional
 2019-02-26 13:50:05 -08:00
Ganesh Vernekar 28c73f531f
Make vertical compaction and query merge optional 
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-26 11:50:50 -08:00
Julius Volz 752e022aba Add "dump" command to tsdb tool to dump all samples (#532) 
* Add "dump" command to tsdb tool to dump all samples

Signed-off-by: Julius Volz <julius.volz@gmail.com>
 2019-02-25 15:51:33 +02:00
Goutham Veeramachaneni 10d395259b Avoid creation of 0 sized segments. (#527) 
If the corrupt segment is full, then we set donePages on open,
c59ed492b2/wal/wal.go (L235-L243)

Then when we try to repair, we set the segment to be a new segment but
we don't update the donePages: c59ed492b2/wal/wal.go (L334)

We we try to log to this, because donePages is full, we will never log
anything to this segment and create a new one: c59ed492b2/wal/wal.go (L486)

This does not cause issues because we simply concatenate the segments on
read, there by transparently skipping this `0b` segment.
 2019-02-25 12:10:27 +02:00
Alec b48394eeb3 Add test for be64() in encoding/encoding.go (#521) (#533) 
* Add test for be64() in encoding/encoding.go

Signed-off-by: naivewong <867245430@qq.com>
 2019-02-23 11:51:47 +02:00
Alec e7436e13f0 Merge encoding_helpers.go to tsdbutil (#526) 
remove duplicate encoding helper funcs and move to own package so they can be reused.

Signed-off-by: naivewong <867245430@qq.com>
 2019-02-22 19:11:11 +02:00
Tom Wilkie 77d5a7d47a
LiveReader can get into an infinite loop on corrupt WALs. (#524) 
Make WAL live tailer return EOF when the there is a half-written record at the end of the file.

Previously, this would cause an infinite loop as we ignored EOFs when filling the buffer.  We now differentiate between EOFs that read >0 bytes, and EOFs that didn't.

Add some more unit tests for tailing a corrupt WAL, and unify interfaces Reader and LiveReader for the purposes of testing.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
 2019-02-19 14:33:57 +00:00
Krasi Georgiev 8913617a9e
update makefile.common and run make all be default. (#529) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-19 13:55:01 +02:00
Tom Wilkie bc3b0bd429
Test to corrupt segments mid-WAL, repair and check we can read the correct number of records. (#528) 
Test to corrupt segments mid-WAL, repair and check we can read the correct number of records.

Make segmentBufReader pad short segments with zeros, and only advance curr segment index after fully reading segment.
 2019-02-18 19:05:07 +00:00
Ganesh Vernekar c59ed492b2 Vertical query merging and compaction (#370) 
* Vertical series iterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Select overlapped blocks first in compactor Plan()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added vertical compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Code cleanup and comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add benchmark for compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Perform vertical compaction only when blocks are overlapping.

Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for vertical compaction

* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for query iterator and seek for non overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Vertical query merge only for overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Simplify logging in Compact(...)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Calculate overlapping inside populateBlock

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* MinTime and MaxTime for BlockReader.

Using this to find overlapping blocks in populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Sort blocks w.r.t. MinTime in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping in LeveledCompactor.write() instead of returning bool

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping inside LeveledCompactor.populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor createBlock to take optional []Series

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* review1

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* Updated CHANGELOG and minor nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor iterator and seek benchmarks for Querier.

Also has as overlapping blocks.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Additional test case

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Split genSeries into genSeries and populateSeries

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Check error in benchmark

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Warn about overlapping blocks in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-14 14:29:41 +01:00
Callum Styan 89ee5aaed4 clarify which segments are deleted when we find a corrupted segment (#522) 
Signed-off-by: Callum Styan <callumstyan@gmail.com>
 2019-02-14 05:44:19 +02:00
Bryan Boreham 74093f0508 Remove pointer indirection on chunk bstream (#499) 
It isn't necessary since the member is never changed after
initialization.

Signed-off-by: Bryan Boreham <bryan@weave.works>
 2019-02-13 23:41:12 +01:00
Alec 109252f3aa Update encoding_helpers.go (len of be64 should be 8) (#521) 2019-02-13 11:04:21 +02:00
Krasi Georgiev 9f28ffa6f4
Merge pull request #465 from krasi-georgiev/shutdown-during-compaction 
use context to cancel compactions
 2019-02-12 11:25:40 +02:00
Krasi Georgiev bf79c767f0 new line 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 11:08:09 +02:00
Krasi Georgiev beee5c58f3 Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 10:56:45 +02:00
Krasi Georgiev 6fce018def
Merge pull request #512 from krasi-georgiev/delete-compact-block-on-reload-error 
Delete compact block on reload error
 2019-02-12 10:50:21 +02:00
Krasi Georgiev bf2239079d refactor multi errors 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 12:28:46 +02:00
Krasi Georgiev c3a5c1d891 refactor error handling 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 12:22:11 +02:00
Krasi Georgiev 07df4fd383 nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 11:25:57 +02:00
Krasi Georgiev 0f8f5027ef remove nested for if 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 18:09:23 +02:00
Krasi Georgiev d48606827c simplify closers 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 13:35:32 +02:00
Krasi Georgiev e138c7ed7e Merge remote-tracking branch 'upstream/master' into delete-compact-block-on-reload-error 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 13:26:28 +02:00
Krasi Georgiev da9da9fbee fix the sleep logic 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 12:39:25 +02:00
Krasi Georgiev 0b72f9af4c
Merge pull request #270 from codesome/master 
Head: don't create stones, delete samples directly
 2019-02-08 12:35:01 +02:00
Krasi Georgiev 1e91d619de
Merge pull request #493 from simonpasquier/update-makefile-common 
fix static check errors and removed a lot of unused code and vars.
updated Makefile.common
 2019-02-08 11:47:06 +02:00
Krasi Georgiev 457534d5c4 simplify nesting. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 11:36:30 +02:00
Ganesh Vernekar 5481549324
Merge remote-tracking branch 'upstream/master' 
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-07 22:15:06 +05:30
Simon Pasquier 272fd0eabf Update Makefile.common 
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
 2019-02-07 15:06:10 +01:00
Simon Pasquier 95334f13c5 Merge branch 'master' into update-makefile-common 
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
 2019-02-07 12:10:22 +01:00
Krasi Georgiev 2ae0620205 rename some vars and use Gauge instead of Counter for metrics 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-07 10:09:42 +02:00
Krasi Georgiev 776769377e fix merr logic. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-06 16:59:28 +02:00
Krasi Georgiev 08e7bc8ee8 always remove tmp 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-06 16:09:42 +02:00
Krasi Georgiev 45acaadd81 review changes 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-06 14:07:35 +02:00
Krasi Georgiev 77cc2ea210
Update index.md(format of series entry) (#515) 
* Update index.md
* more clear format representation
* move dots to the bottom
* update #labels count, #chunks count
* use correct variable types.

Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 12:13:38 +03:00
naivewong 9b227d27e7 update labels count, chunks count 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 10:00:18 +08:00
naivewong 8e7e2041d3 update #labels count, #chunks count 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 00:48:25 +08:00
naivewong 42ee59689b move dots to the bottom 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 00:34:15 +08:00
naivewong 0649dfddf0 more clear format representation 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-06 00:05:30 +08:00
naivewong 1f723a8eb5 update #labels and #chunks 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-05 23:20:43 +08:00
Alec 8e589474c6 Update index.md 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-05 10:34:25 +08:00
Alec 2b6bc9fb32 Update index.md 
Signed-off-by: naivewong <867245430@qq.com>
 2019-02-05 10:34:25 +08:00
Alec 1bcda9d23f Update tombstones.md (format of tombstone) (#511) 2019-02-04 12:30:47 +03:00
Krasi Georgiev ce4a2083fb nit 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-04 11:15:08 +02:00
Krasi Georgiev 752ab86e4e change the test block series for more stable tests 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-04 11:14:39 +02:00
Krasi Georgiev 00d5e19baf Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 13:47:50 +02:00
Krasi Georgiev 6c34eb8b63 nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 12:17:15 +02:00
Krasi Georgiev 315de4c782 fix windows tests 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 11:40:40 +02:00