mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-28 06:59:40 -08:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity
1011 commits 243 branches 444 tags 649 MiB
Commit graph

1011 commits

This branch
This branch
All branches
Author SHA1 Message Date
Krasi Georgiev 8eeb70fee1
remove Fsync workaround for macos. (#574) 
since golang 1.12 no special handling is required for file.Sync()

@pborzenkov thanks for the pointer.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-04-03 11:16:54 +03:00
Pavel Borzenkov 4f204dcbc1 chunks: close all opened MmapFiles on NewDirReader error path (#575) 
Signed-off-by: Pavel Borzenkov <pavel.borzenkov@gmail.com>
 2019-04-02 15:16:29 +03:00
Krasi Georgiev e46ec89de6
use fsync to force persisting the meta file. (#573) 
* use fsync to force persisting the meta file.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-04-01 16:23:45 +03:00
zhulongcheng aed16621c0 Add Head.compactable method (#542) 
* Add Head.compactable method

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-04-01 11:19:06 +03:00
Vladimir Masarik fd188be052 Fixed default benchmark test path (#571) 
Signed-off-by: Vladimir Masarik <vmasarik@redhat.com>
 2019-03-27 16:33:59 +02:00
Vishnunarayan K I 7757fe6f21 Added ability to create db with NopWal (#519) 
Signed-off-by: Vishnunarayan K I <appukuttancr@gmail.com>
 2019-03-26 01:38:12 +02:00
zhulongcheng db9177de0c docs: add multi chunks layout in chunks readme file (#567) 2019-03-25 12:33:38 +02:00
zhulongcheng e11e01f68d Fix exported function comments (#566) 
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-03-25 10:17:28 +02:00
zhulongcheng 4d03c70800 Verify chunk format version (#544) 
Verify chunk format version.
 2019-03-24 22:33:08 +02:00
Bartek Płotka 3ab5f4e579 index: reduce empty postings trees (#509) 
Improved Merge when all is empty.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2019-03-21 18:23:00 +02:00
naivewong 1c392d0057 Update wal.md that the first wal entry includes only base id and timestamp without a value (#562) 
Signed-off-by: naivewong <867245430@qq.com>
 2019-03-21 11:21:50 +02:00
zhulongcheng 837ae9aaa0 Update comment for ErrOutOfOrderSample (#563) 
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-03-21 10:53:39 +02:00
zhulongcheng 62cfe4446f Make Head.symbols map with size hint (#552) 
To reduce the number of times the map is resized

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-03-20 10:43:07 +02:00
zhulongcheng 921da4dced Remove unused statement from index test (#558) 
Remove unused `i++` statement from index test.
 2019-03-19 15:41:40 +02:00
Krasi Georgiev c3ffdf1a99
Test createBlock and check all os.RemoveAll in the tests for errors. (#549) 
Testing that createBlock creates blocks that can be opened.

and checking the os.RemoveAll for errors will catch errors for un-closed files under windows.

Many missing .Close() calls were added for fixing failing os.RemoveAll

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-03-19 15:31:57 +02:00
naivewong dd0d3c6f02 Several small fixes (#550) 
`if ds[i].meta.MinTime < t0 || ds[i].meta.MaxTime > t0+tr `, ds[i].meta.MinTime is always larger or equal to t0, so no need for this check.
`ulid.Parse` only checks if the length is 26. So changed to  using `ulid.ParseStrict` to also check the validity of ulid.
 2019-03-18 16:14:10 +02:00
zhulongcheng 99703b3269 README: move the build badge to the badge list (#555) 
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-03-18 14:44:58 +02:00
zhulongcheng 95648b33c4 Fix a typo in head.go (#553) 
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-03-18 10:37:20 +02:00
zhulongcheng 8feb67e630 Update Appender.AddFast method comment (#551) 
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-03-17 07:42:18 +02:00
zhulongcheng b7cac504c9 Remove unused NewReader function (#547) 
The `NewReader` function is not used anywhere.
 2019-03-14 15:05:40 +02:00
Simon Pasquier 84e51f7222 Update Makefile.common (#548) 
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
 2019-03-13 10:40:17 +02:00
zhulongcheng 719b57db44 Remove obsolete FromData function (#540) 
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-03-06 11:12:28 +02:00
Ganesh Vernekar 8921f18eb3
Merge pull request #543 from codesome/v0.6.1 
*:cut 0.6.1
 2019-03-05 21:36:06 +05:30
Ganesh Vernekar fcfe312d99
Update CHANGELOG for v0.6.1 
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-03-05 19:21:46 +05:30
Alec 0861a9b14f [bugfix] update 'last' variable in chunks.MergeOverlappingChunks() (#539) 
* update

Signed-off-by: naivewong <867245430@qq.com>

* update comment

Signed-off-by: naivewong <867245430@qq.com>

* Update failing test.

Signed-off-by: naivewong <867245430@qq.com>

* Update comment

Signed-off-by: naivewong <867245430@qq.com>

* Add changelog entry.

Signed-off-by: naivewong <867245430@qq.com>

* Update CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-03-05 19:18:55 +05:30
Simon Pasquier 08af7bbf90 Add Go 1.12 to Travis CI (#538) 
Also update to the latest version of Makefile.common.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
 2019-03-04 21:44:40 +02:00
Krasi Georgiev ab8476dd06
Release 0.6.0 (#537) 2019-03-02 15:54:49 +02:00
Krasi Georgiev e78fedf054
release 0.5.0 (#536) 
* release 0.5.0

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-03-01 22:57:53 +02:00
Brian Brazil 62b652fbd0
Improve Merge performance (#531) 
Use a heap for Next for merges, and
pre-compute if there's many postings on the
unset path.

Add posting lookup benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2019-02-28 17:23:55 +00:00
Krasi Georgiev df06f9ebc2
update the CHANGELOG with some missing changes. (#535) 
missed to add these with some merged PRs

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-27 12:55:35 +02:00
Ganesh Vernekar 158c3074cd
Merge pull request #534 from codesome/optional-vertical-blocks 
Make vertical compaction and query merge optional
 2019-02-26 13:50:05 -08:00
Ganesh Vernekar 28c73f531f
Make vertical compaction and query merge optional 
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-26 11:50:50 -08:00
Julius Volz 752e022aba Add "dump" command to tsdb tool to dump all samples (#532) 
* Add "dump" command to tsdb tool to dump all samples

Signed-off-by: Julius Volz <julius.volz@gmail.com>
 2019-02-25 15:51:33 +02:00
Goutham Veeramachaneni 10d395259b Avoid creation of 0 sized segments. (#527) 
If the corrupt segment is full, then we set donePages on open,
c59ed492b2/wal/wal.go (L235-L243)

Then when we try to repair, we set the segment to be a new segment but
we don't update the donePages: c59ed492b2/wal/wal.go (L334)

We we try to log to this, because donePages is full, we will never log
anything to this segment and create a new one: c59ed492b2/wal/wal.go (L486)

This does not cause issues because we simply concatenate the segments on
read, there by transparently skipping this `0b` segment.
 2019-02-25 12:10:27 +02:00
Alec b48394eeb3 Add test for be64() in encoding/encoding.go (#521) (#533) 
* Add test for be64() in encoding/encoding.go

Signed-off-by: naivewong <867245430@qq.com>
 2019-02-23 11:51:47 +02:00
Alec e7436e13f0 Merge encoding_helpers.go to tsdbutil (#526) 
remove duplicate encoding helper funcs and move to own package so they can be reused.

Signed-off-by: naivewong <867245430@qq.com>
 2019-02-22 19:11:11 +02:00
Tom Wilkie 77d5a7d47a
LiveReader can get into an infinite loop on corrupt WALs. (#524) 
Make WAL live tailer return EOF when the there is a half-written record at the end of the file.

Previously, this would cause an infinite loop as we ignored EOFs when filling the buffer.  We now differentiate between EOFs that read >0 bytes, and EOFs that didn't.

Add some more unit tests for tailing a corrupt WAL, and unify interfaces Reader and LiveReader for the purposes of testing.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
 2019-02-19 14:33:57 +00:00
Krasi Georgiev 8913617a9e
update makefile.common and run make all be default. (#529) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-19 13:55:01 +02:00
Tom Wilkie bc3b0bd429
Test to corrupt segments mid-WAL, repair and check we can read the correct number of records. (#528) 
Test to corrupt segments mid-WAL, repair and check we can read the correct number of records.

Make segmentBufReader pad short segments with zeros, and only advance curr segment index after fully reading segment.
 2019-02-18 19:05:07 +00:00
Ganesh Vernekar c59ed492b2 Vertical query merging and compaction (#370) 
* Vertical series iterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Select overlapped blocks first in compactor Plan()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added vertical compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Code cleanup and comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add benchmark for compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Perform vertical compaction only when blocks are overlapping.

Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for vertical compaction

* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for query iterator and seek for non overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Vertical query merge only for overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Simplify logging in Compact(...)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Calculate overlapping inside populateBlock

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* MinTime and MaxTime for BlockReader.

Using this to find overlapping blocks in populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Sort blocks w.r.t. MinTime in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping in LeveledCompactor.write() instead of returning bool

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping inside LeveledCompactor.populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor createBlock to take optional []Series

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* review1

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* Updated CHANGELOG and minor nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor iterator and seek benchmarks for Querier.

Also has as overlapping blocks.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Additional test case

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Split genSeries into genSeries and populateSeries

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Check error in benchmark

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Warn about overlapping blocks in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-14 14:29:41 +01:00
Callum Styan 89ee5aaed4 clarify which segments are deleted when we find a corrupted segment (#522) 
Signed-off-by: Callum Styan <callumstyan@gmail.com>
 2019-02-14 05:44:19 +02:00
Bryan Boreham 74093f0508 Remove pointer indirection on chunk bstream (#499) 
It isn't necessary since the member is never changed after
initialization.

Signed-off-by: Bryan Boreham <bryan@weave.works>
 2019-02-13 23:41:12 +01:00
Alec 109252f3aa Update encoding_helpers.go (len of be64 should be 8) (#521) 2019-02-13 11:04:21 +02:00
Krasi Georgiev 9f28ffa6f4
Merge pull request #465 from krasi-georgiev/shutdown-during-compaction 
use context to cancel compactions
 2019-02-12 11:25:40 +02:00
Krasi Georgiev bf79c767f0 new line 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 11:08:09 +02:00
Krasi Georgiev beee5c58f3 Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 10:56:45 +02:00
Krasi Georgiev 6fce018def
Merge pull request #512 from krasi-georgiev/delete-compact-block-on-reload-error 
Delete compact block on reload error
 2019-02-12 10:50:21 +02:00
Krasi Georgiev bf2239079d refactor multi errors 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 12:28:46 +02:00
Krasi Georgiev c3a5c1d891 refactor error handling 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 12:22:11 +02:00
Krasi Georgiev 07df4fd383 nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 11:25:57 +02:00