prometheus/tsdb
Chris Marchbanks 8df4bca470
Garbage collect asynchronously in the WAL Watcher
The WAL Watcher replays a checkpoint after it is created in order to
garbage collect series that no longer exist in the WAL. Currently the
garbage collection process is done serially with reading from the tip of
the WAL which can cause large delays in writing samples to remote
storage just after compaction occurs.

This also fixes a memory leak where dropped series are not cleaned up as
part of the SeriesReset process.

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2019-10-07 14:36:10 -06:00
..
chunkenc Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
chunks Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
cmd/tsdb Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
docs/format tsdb: update chunks format (#6033) 2019-09-19 13:56:32 +03:00
encoding Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
errors Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
fileutil Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
goversion fix the wrong word (#6069) 2019-09-30 09:54:55 -06:00
index Add test for MemPostings.Delete (#5910) 2019-08-22 15:19:12 +03:00
labels Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
record Move WAL watcher code to tsdb/wal package. (#5999) 2019-09-19 14:45:41 +05:30
test Cleanup after merging tsdb into prometheus 2019-08-13 14:04:14 +05:30
testdata Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
tombstones Move WAL watcher code to tsdb/wal package. (#5999) 2019-09-19 14:45:41 +05:30
tsdbutil Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
wal Garbage collect asynchronously in the WAL Watcher 2019-10-07 14:36:10 -06:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go fix the wrong word (#6069) 2019-09-30 09:54:55 -06:00
block_test.go Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
CHANGELOG.md fix-up tsdb-typo (#5954) 2019-08-28 14:43:02 +01:00
compact.go Move WAL watcher code to tsdb/wal package. (#5999) 2019-09-19 14:45:41 +05:30
compact_test.go fix the wrong word (#6069) 2019-09-30 09:54:55 -06:00
db.go fix the wrong word (#6069) 2019-09-30 09:54:55 -06:00
db_test.go Move WAL watcher code to tsdb/wal package. (#5999) 2019-09-19 14:45:41 +05:30
head.go Allocate the shards only once while reading WAL (#6093) 2019-10-03 15:21:39 +05:30
head_bench_test.go Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
head_test.go Benchmark for loading WAL (#6081) 2019-10-03 11:49:55 +05:30
MAINTAINERS.md Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
mocks_test.go Move WAL watcher code to tsdb/wal package. (#5999) 2019-09-19 14:45:41 +05:30
querier.go fix the wrong word (#6069) 2019-09-30 09:54:55 -06:00
querier_test.go Move WAL watcher code to tsdb/wal package. (#5999) 2019-09-19 14:45:41 +05:30
README.md Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
repair.go tsdb: add block meta version constant (#5994) 2019-09-09 12:28:01 +03:00
repair_test.go tsdb: add block meta version constant (#5994) 2019-09-09 12:28:01 +03:00
wal.go fix the wrong word (#6069) 2019-09-30 09:54:55 -06:00
wal_test.go fix the wrong word (#6069) 2019-09-30 09:54:55 -06:00

TSDB

Build Status GoDoc Go Report Card

This repository contains the Prometheus storage layer that is used in its 2.x releases.

A writeup of its design can be found here.

Based on the Gorilla TSDB white papers.

Video: Storing 16 Bytes at Scale from PromCon 2017.

See also the format documentation.