Commit graph

994 commits

Author SHA1 Message Date
zhulongcheng 95648b33c4 Fix a typo in head.go (#553)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-18 10:37:20 +02:00
zhulongcheng 8feb67e630 Update Appender.AddFast method comment (#551)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-17 07:42:18 +02:00
zhulongcheng b7cac504c9 Remove unused NewReader function (#547)
The `NewReader` function is not used anywhere.
2019-03-14 15:05:40 +02:00
Simon Pasquier 84e51f7222 Update Makefile.common (#548)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-03-13 10:40:17 +02:00
zhulongcheng 719b57db44 Remove obsolete FromData function (#540)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-03-06 11:12:28 +02:00
Ganesh Vernekar 8921f18eb3
Merge pull request #543 from codesome/v0.6.1
*:cut 0.6.1
2019-03-05 21:36:06 +05:30
Ganesh Vernekar fcfe312d99
Update CHANGELOG for v0.6.1
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-03-05 19:21:46 +05:30
Alec 0861a9b14f [bugfix] update 'last' variable in chunks.MergeOverlappingChunks() (#539)
* update

Signed-off-by: naivewong <867245430@qq.com>

* update comment

Signed-off-by: naivewong <867245430@qq.com>

* Update failing test.

Signed-off-by: naivewong <867245430@qq.com>

* Update comment

Signed-off-by: naivewong <867245430@qq.com>

* Add changelog entry.

Signed-off-by: naivewong <867245430@qq.com>

* Update CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-03-05 19:18:55 +05:30
Simon Pasquier 08af7bbf90 Add Go 1.12 to Travis CI (#538)
Also update to the latest version of Makefile.common.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-03-04 21:44:40 +02:00
Krasi Georgiev ab8476dd06
Release 0.6.0 (#537) 2019-03-02 15:54:49 +02:00
Krasi Georgiev e78fedf054
release 0.5.0 (#536)
* release 0.5.0

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-03-01 22:57:53 +02:00
Brian Brazil 62b652fbd0
Improve Merge performance (#531)
Use a heap for Next for merges, and
pre-compute if there's many postings on the
unset path.

Add posting lookup benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-02-28 17:23:55 +00:00
Krasi Georgiev df06f9ebc2
update the CHANGELOG with some missing changes. (#535)
missed to add these with some merged PRs

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-27 12:55:35 +02:00
Ganesh Vernekar 158c3074cd
Merge pull request #534 from codesome/optional-vertical-blocks
Make vertical compaction and query merge optional
2019-02-26 13:50:05 -08:00
Ganesh Vernekar 28c73f531f
Make vertical compaction and query merge optional
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-02-26 11:50:50 -08:00
Julius Volz 752e022aba Add "dump" command to tsdb tool to dump all samples (#532)
* Add "dump" command to tsdb tool to dump all samples

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-25 15:51:33 +02:00
Goutham Veeramachaneni 10d395259b Avoid creation of 0 sized segments. (#527)
If the corrupt segment is full, then we set donePages on open,
c59ed492b2/wal/wal.go (L235-L243)

Then when we try to repair, we set the segment to be a new segment but
we don't update the donePages: c59ed492b2/wal/wal.go (L334)

We we try to log to this, because donePages is full, we will never log
anything to this segment and create a new one: c59ed492b2/wal/wal.go (L486)

This does not cause issues because we simply concatenate the segments on
read, there by transparently skipping this `0b` segment.
2019-02-25 12:10:27 +02:00
Alec b48394eeb3 Add test for be64() in encoding/encoding.go (#521) (#533)
* Add test for be64() in encoding/encoding.go

Signed-off-by: naivewong <867245430@qq.com>
2019-02-23 11:51:47 +02:00
Alec e7436e13f0 Merge encoding_helpers.go to tsdbutil (#526)
remove duplicate encoding helper funcs and move to own package so they can be reused.

Signed-off-by: naivewong <867245430@qq.com>
2019-02-22 19:11:11 +02:00
Tom Wilkie 77d5a7d47a
LiveReader can get into an infinite loop on corrupt WALs. (#524)
Make WAL live tailer return EOF when the there is a half-written record at the end of the file.

Previously, this would cause an infinite loop as we ignored EOFs when filling the buffer.  We now differentiate between EOFs that read >0 bytes, and EOFs that didn't.

Add some more unit tests for tailing a corrupt WAL, and unify interfaces Reader and LiveReader for the purposes of testing.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-19 14:33:57 +00:00
Krasi Georgiev 8913617a9e
update makefile.common and run make all be default. (#529)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-19 13:55:01 +02:00
Tom Wilkie bc3b0bd429
Test to corrupt segments mid-WAL, repair and check we can read the correct number of records. (#528)
Test to corrupt segments mid-WAL, repair and check we can read the correct number of records.

Make segmentBufReader pad short segments with zeros, and only advance curr segment index after fully reading segment.
2019-02-18 19:05:07 +00:00
Ganesh Vernekar c59ed492b2 Vertical query merging and compaction (#370)
* Vertical series iterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Select overlapped blocks first in compactor Plan()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added vertical compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Code cleanup and comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add benchmark for compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Perform vertical compaction only when blocks are overlapping.

Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for vertical compaction

* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for query iterator and seek for non overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Vertical query merge only for overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Simplify logging in Compact(...)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Calculate overlapping inside populateBlock

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* MinTime and MaxTime for BlockReader.

Using this to find overlapping blocks in populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Sort blocks w.r.t. MinTime in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping in LeveledCompactor.write() instead of returning bool

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping inside LeveledCompactor.populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor createBlock to take optional []Series

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* review1

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* Updated CHANGELOG and minor nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor iterator and seek benchmarks for Querier.

Also has as overlapping blocks.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Additional test case

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Split genSeries into genSeries and populateSeries

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Check error in benchmark

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Warn about overlapping blocks in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-02-14 14:29:41 +01:00
Callum Styan 89ee5aaed4 clarify which segments are deleted when we find a corrupted segment (#522)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-02-14 05:44:19 +02:00
Bryan Boreham 74093f0508 Remove pointer indirection on chunk bstream (#499)
It isn't necessary since the member is never changed after
initialization.

Signed-off-by: Bryan Boreham <bryan@weave.works>
2019-02-13 23:41:12 +01:00
Alec 109252f3aa Update encoding_helpers.go (len of be64 should be 8) (#521) 2019-02-13 11:04:21 +02:00
Krasi Georgiev 9f28ffa6f4
Merge pull request #465 from krasi-georgiev/shutdown-during-compaction
use context to cancel compactions
2019-02-12 11:25:40 +02:00
Krasi Georgiev bf79c767f0 new line
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-12 11:08:09 +02:00
Krasi Georgiev beee5c58f3 Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-12 10:56:45 +02:00
Krasi Georgiev 6fce018def
Merge pull request #512 from krasi-georgiev/delete-compact-block-on-reload-error
Delete compact block on reload error
2019-02-12 10:50:21 +02:00
Krasi Georgiev bf2239079d refactor multi errors
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-11 12:28:46 +02:00
Krasi Georgiev c3a5c1d891 refactor error handling
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-11 12:22:11 +02:00
Krasi Georgiev 07df4fd383 nits
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-11 11:25:57 +02:00
Krasi Georgiev 0f8f5027ef remove nested for if
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-08 18:09:23 +02:00
Krasi Georgiev d48606827c simplify closers
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-08 13:35:32 +02:00
Krasi Georgiev e138c7ed7e Merge remote-tracking branch 'upstream/master' into delete-compact-block-on-reload-error
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-08 13:26:28 +02:00
Krasi Georgiev da9da9fbee fix the sleep logic
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-08 12:39:25 +02:00
Krasi Georgiev 0b72f9af4c
Merge pull request #270 from codesome/master
Head: don't create stones, delete samples directly
2019-02-08 12:35:01 +02:00
Krasi Georgiev 1e91d619de
Merge pull request #493 from simonpasquier/update-makefile-common
fix static check errors and removed a lot of unused code and vars.
updated Makefile.common
2019-02-08 11:47:06 +02:00
Krasi Georgiev 457534d5c4 simplify nesting.
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-08 11:36:30 +02:00
Ganesh Vernekar 5481549324
Merge remote-tracking branch 'upstream/master'
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-02-07 22:15:06 +05:30
Simon Pasquier 272fd0eabf Update Makefile.common
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-07 15:06:10 +01:00
Simon Pasquier 95334f13c5 Merge branch 'master' into update-makefile-common
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-07 12:10:22 +01:00
Krasi Georgiev 2ae0620205 rename some vars and use Gauge instead of Counter for metrics
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-07 10:09:42 +02:00
Krasi Georgiev 776769377e fix merr logic.
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-06 16:59:28 +02:00
Krasi Georgiev 08e7bc8ee8 always remove tmp
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-06 16:09:42 +02:00
Krasi Georgiev 45acaadd81 review changes
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-06 14:07:35 +02:00
Krasi Georgiev 77cc2ea210
Update index.md(format of series entry) (#515)
* Update index.md
* more clear format representation
* move dots to the bottom
* update #labels count, #chunks count
* use correct variable types.

Signed-off-by: naivewong <867245430@qq.com>
2019-02-06 12:13:38 +03:00
naivewong 9b227d27e7 update labels count, chunks count
Signed-off-by: naivewong <867245430@qq.com>
2019-02-06 10:00:18 +08:00
naivewong 8e7e2041d3 update #labels count, #chunks count
Signed-off-by: naivewong <867245430@qq.com>
2019-02-06 00:48:25 +08:00