prometheus/tsdb
Bartlomiej Plotka e6d7cc5fa4
tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069)
* tsdb: Added ChunkQueryable implementations to db; unified compactor, querier and fanout block iterating.

Chained to https://github.com/prometheus/prometheus/pull/7059

* NewMerge(Chunk)Querier now takies multiple primaries allowing tsdb DB code to use it.
* Added single SeriesEntry / ChunkEntry for all series implementations.
* Unified all vertical, and non vertical for compact and querying to single
merge series / chunk sets by reusing VerticalSeriesMergeFunc for overlapping algorithm (same logic as before)
* Added block (Base/Chunk/)Querier for block querying. We then use populateAndTomb(Base/Chunk/) to iterate over chunks or samples.
* Refactored endpoint tests and querier tests to include subtests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed comments from Brian and Beorn.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed snapshot test and added chunk iterator support for DBReadOnly.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed race when iterating over Ats first.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed populate block tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed endpoints test.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed test.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added test & fixed case of head open chunk.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed DBReadOnly tests and bug producing 1 sample chunks.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added cases for partial block overlap for multiple full chunks.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added extra tests for chunk meta after compaction.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed small vertical merge bug and added more tests for that.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-07-31 16:03:02 +01:00
..
chunkenc Cleanup bstream_test.go based on post-merge feedback received on #7390 (#7413) 2020-06-18 14:49:39 +05:30
chunks tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
docs/format tsdb/docs: fix head chunks directory + link from README (#7309) 2020-06-17 20:38:21 +05:30
encoding Avoid WriteAt for Postings. 2019-12-16 15:30:49 +00:00
errors Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
fileutil Avoid empty mmap files by using .tmp files to write headers 2020-07-14 14:59:28 +05:30
goversion tsdb: test for leaks (#7566) 2020-07-21 10:08:06 +02:00
index tsdb: test for leaks (#7566) 2020-07-21 10:08:06 +02:00
record Add unit test case for func Type in record.go (#7082) 2020-05-27 12:08:33 +05:30
test Port tsdb to use pkg/labels. (#6326) 2019-11-18 11:53:33 -08:00
testdata Handle V1 indexes, some of which have unsorted posting offset tables. (#6564) 2020-01-06 14:06:11 +00:00
tombstones tsdb: test for leaks (#7566) 2020-07-21 10:08:06 +02:00
tsdbutil tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
wal tsdb: test for leaks (#7566) 2020-07-21 10:08:06 +02:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
block_test.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
CHANGELOG.md Spelling (#6517) 2020-01-02 15:54:09 +01:00
compact.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
compact_test.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
db.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
db_test.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
head.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
head_bench_test.go Replace sync/atomic with uber-go/atomic (#7683) 2020-07-30 13:15:42 +05:30
head_test.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
isolation.go Optimise lowWatermark in Isolation (#7332) 2020-06-03 20:09:05 +02:00
isolation_test.go Optimise lowWatermark in Isolation (#7332) 2020-06-03 20:09:05 +02:00
mocks_test.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
querier.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
querier_bench_test.go Fix failing tests 2020-07-30 17:06:56 +05:30
querier_test.go tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) 2020-07-31 16:03:02 +01:00
README.md Edit TSDB README badges 2019-10-24 15:35:47 +05:30
repair.go Capitalizing first letter of all log lines (#7043) 2020-04-11 09:22:18 +01:00
repair_test.go Port tsdb to use pkg/labels. (#6326) 2019-11-18 11:53:33 -08:00
tsdbblockutil.go Address comments 2020-07-30 16:41:13 +05:30
wal.go Capitalizing first letter of all log lines (#7043) 2020-04-11 09:22:18 +01:00
wal_test.go tsdb: test for leaks (#7566) 2020-07-21 10:08:06 +02:00

TSDB

GoDoc

This repository contains the Prometheus storage layer that is used in its 2.x releases.

A writeup of its design can be found here.

Based on the Gorilla TSDB white papers.

Video: Storing 16 Bytes at Scale from PromCon 2017.

See also the format documentation.