prometheus/tsdb
Bryan Boreham 1b74a3812e
Fix panic, out of order chunks, and race warning during WAL replay (#9856)
* Fix panic on WAL replay

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Refactor: introduce walSubsetProcessor

walSubsetProcessor packages up the `processWALSamples()` function and
its input and output channels, helping to clarify how these things
relate.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Refactor: extract more methods onto walSubsetProcessor

This makes the main logic easier to follow.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Fix race warning by locking processWALSamples

Although we have waited for the processor to finish, we still get a
warning from the race detector because it doesn't know how the different
parts relate.

Add a lock round each batch of samples, so the race detector can see
that we never access series owned by the processor outside of a lock.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Added test to reproduce issue 9859

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Remove redundant unit test

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix out of order chunks during WAL replay

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix nits

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Co-authored-by: Marco Pracucci <marco@pracucci.com>
2021-11-25 13:36:14 +05:30
..
agent Share TSDB locker code with agent (#9623) 2021-11-11 11:45:25 -05:00
chunkenc improve bstream comments and doc (#9560) 2021-10-25 18:44:15 +05:30
chunks clarify HeadChunkID type and usage (#9726) 2021-11-17 18:35:10 +05:30
docs clarify HeadChunkID type and usage (#9726) 2021-11-17 18:35:10 +05:30
encoding Format Go source files using 'gofumpt -w -s -extra' 2021-11-02 19:52:34 +01:00
errors Address golint failures from revive 2021-10-23 00:53:11 +02:00
fileutil tsdb: Add windows arm64 support. 2021-11-09 11:07:27 +01:00
goversion Add new Go build tags. 2021-08-27 10:24:14 +02:00
index Check postings bytes length when decoding (#9766) 2021-11-24 15:26:37 +05:30
record Add basic initial developer docs for TSDB (#9451) 2021-11-17 15:51:27 +05:30
test Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
testdata tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 2020-08-11 06:56:08 +01:00
tombstones TSDB: demistify SeriesRefs and ChunkRefs (#9536) 2021-11-06 15:40:04 +05:30
tsdbutil Share TSDB locker code with agent (#9623) 2021-11-11 11:45:25 -05:00
wal fix: panic when checkpoint directory is empty (#9687) 2021-11-17 16:39:04 +05:30
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
block_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
blockwriter.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
blockwriter_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact.go TSDB: demistify SeriesRefs and ChunkRefs (#9536) 2021-11-06 15:40:04 +05:30
compact_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
db.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
db_test.go Fix panic, out of order chunks, and race warning during WAL replay (#9856) 2021-11-25 13:36:14 +05:30
example_test.go Add basic initial developer docs for TSDB (#9451) 2021-11-17 15:51:27 +05:30
exemplar.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
exemplar_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
head.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
head_append.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
head_bench_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
head_read.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
head_test.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
head_wal.go Fix panic, out of order chunks, and race warning during WAL replay (#9856) 2021-11-25 13:36:14 +05:30
isolation.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
isolation_test.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
mocks_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
querier.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
querier_bench_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
querier_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
README.md Add basic initial developer docs for TSDB (#9451) 2021-11-17 15:51:27 +05:30
repair.go Switched to go-kit/log 2021-06-11 12:28:36 -04:00
repair_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
tsdbblockutil.go TSDB: demistify SeriesRefs and ChunkRefs (#9536) 2021-11-06 15:40:04 +05:30
wal.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
wal_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Due to an issue with versioning, the "latest" docs shown on Godoc are outdated. Instead you may use the docs for v2.31.1

Documentation

External resources

A series of blog posts explaining different components of TSDB: