mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-24 21:24:05 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
prometheus/CHANGELOG.md
Ganesh Vernekar c59ed492b2 Vertical query merging and compaction (#370) 
* Vertical series iterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Select overlapped blocks first in compactor Plan()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added vertical compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Code cleanup and comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add benchmark for compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Perform vertical compaction only when blocks are overlapping.

Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for vertical compaction

* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for query iterator and seek for non overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Vertical query merge only for overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Simplify logging in Compact(...)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Calculate overlapping inside populateBlock

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* MinTime and MaxTime for BlockReader.

Using this to find overlapping blocks in populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Sort blocks w.r.t. MinTime in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping in LeveledCompactor.write() instead of returning bool

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping inside LeveledCompactor.populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor createBlock to take optional []Series

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* review1

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* Updated CHANGELOG and minor nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor iterator and seek benchmarks for Querier.

Also has as overlapping blocks.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Additional test case

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Split genSeries into genSeries and populateSeries

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Check error in benchmark

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Warn about overlapping blocks in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-02-14 14:29:41 +01:00

2.8 KiB
Raw Blame History

master / unreleased

  • [ENHANCEMENT] Time-ovelapping blocks are now allowed. #370
    • Added MergeChunks function in chunkenc/xor.go to merge 2 time-overlapping chunks.
    • Added MergeOverlappingChunks function in chunks/chunks.go to merge multiple time-overlapping Chunk Metas.
    • Added MinTime and MaxTime method for BlockReader.
  • [CHANGE] NewLeveledCompactor takes a context so that a compaction is canceled when closing the db.
  • [ENHANCEMENT] When closing the db any running compaction will be cancelled so it doesn't block.
  • [CHANGE] prometheus_tsdb_storage_blocks_bytes_total is now prometheus_tsdb_storage_blocks_bytes

0.4.0

  • [CHANGE] New WALSegmentSize option to override the DefaultOptions.WALSegmentSize. Added to allow using smaller wal files. For example using tmpfs on a RPI to minimise the SD card wear out from the constant WAL writes. As part of this change the DefaultOptions.WALSegmentSize constant was also exposed.
  • [CHANGE] Empty blocks are not written during compaction #374
  • [FEATURE] Size base retention through Options.MaxBytes. As part of this change:
    • Added new metrics - prometheus_tsdb_storage_blocks_bytes_total, prometheus_tsdb_size_retentions_total, prometheus_tsdb_time_retentions_total
    • New public interface SizeReader: Size() int64
    • OpenBlock signature changed to take a logger.
  • [REMOVED] PrefixMatcher is considered unused so was removed.
  • [CLEANUP] Options.WALFlushInterval is removed as it wasn't used anywhere.
  • [FEATURE] Add new LiveReader to WAL pacakge. Added to allow live tailing of a WAL segment, used by Prometheus Remote Write after refactor. The main difference between the new reader and the existing Reader is that for LiveReader a call to Next() that returns false does not mean that there will never be more data to read.

0.3.1

  • [BUGFIX] Fixed most windows test and some actual bugs for unclosed file readers.

0.3.0

  • [CHANGE] LastCheckpoint() used to return just the segment name and now it returns the full relative path.
  • [CHANGE] NewSegmentsRangeReader() can now read over miltiple wal ranges by using the new SegmentRange{} struct.
  • [CHANGE] CorruptionErr{} now also exposes the Segment Dir which is added when displaying any errors.
  • [CHANGE] Head.Init() is changed to Head.Init(minValidTime int64)
  • [CHANGE] SymbolTable() renamed to SymbolTableSize() to make the name consistent with the Block{ symbolTableSize uint64 } field.
  • [CHANGE] wal.Reader{} now exposes Segment() for the current segment being read and Offset() for the current offset.
  • [FEATURE] tsdbutil analyze subcomand to find churn, high cardinality, etc.