prometheus/tsdb
Brian Brazil cd73b3d33e
Reduce how much old WAL we keep around. (#7098)
Previously we were keeping up to around 6 hours of WAL around by
removing 1/3 every hours. This was excessive, so switch to removing 2/3
which will up to around 3 hours of WAL around.

This will roughly halve the size of the WAL and halve startup time for
those who are I/O bound. This may increase the checkpoint size for
those with certain churn patterns, but by much less than we're saving
from the segments.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-04-07 15:55:57 +05:30
..
chunkenc storage: Added Chunks{Queryable/Querier/SeriesSet/Series/Iteratable. Added generic Merge{SeriesSet/Querier} implementation. (#7005) 2020-03-24 20:15:47 +00:00
chunks Replace fileutil.ReadDir with ioutil.ReadDir (#7029) (#7033) 2020-04-06 19:04:20 +05:30
cmd/tsdb Use .UTC everywhere we use time.Unix (#7066) 2020-03-29 17:35:39 +01:00
docs/format Live m-mapping of chunks on disk (#6830) 2020-03-19 22:03:44 +05:30
encoding Avoid WriteAt for Postings. 2019-12-16 15:30:49 +00:00
errors Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
fileutil Replace fileutil.ReadDir with ioutil.ReadDir (#7029) (#7033) 2020-04-06 19:04:20 +05:30
goversion Fix some typo in comments (#6730) 2020-01-31 12:11:52 +05:30
index tsdb: writePostingsToTmpFiles returns nil instead of err (#6618) 2020-01-13 22:40:12 +00:00
record Spelling (#6517) 2020-01-02 15:54:09 +01:00
test Port tsdb to use pkg/labels. (#6326) 2019-11-18 11:53:33 -08:00
testdata Handle V1 indexes, some of which have unsorted posting offset tables. (#6564) 2020-01-06 14:06:11 +00:00
tombstones tsdb: add tombstonesHeaderSize constant (#7028) 2020-03-22 12:59:35 +05:30
tsdbutil storage: Added Chunks{Queryable/Querier/SeriesSet/Series/Iteratable. Added generic Merge{SeriesSet/Querier} implementation. (#7005) 2020-03-24 20:15:47 +00:00
wal Replace fileutil.ReadDir with ioutil.ReadDir (#7029) (#7033) 2020-04-06 19:04:20 +05:30
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go Revert head posting optimization 2020-03-25 20:19:33 +01:00
block_test.go storage: Removed SelectSorted method; Simplified interface; Added requirement for remote read to sort response. 2020-03-20 21:14:43 +01:00
CHANGELOG.md Spelling (#6517) 2020-01-02 15:54:09 +01:00
compact.go Merge pull request #7060 from prometheus/release-2.17 2020-03-27 15:57:07 +01:00
compact_test.go Those links should not be reverted 2020-03-25 20:37:26 +01:00
db.go Replace fileutil.ReadDir with ioutil.ReadDir (#7029) (#7033) 2020-04-06 19:04:20 +05:30
db_test.go Those links should not be reverted 2020-03-25 20:37:26 +01:00
head.go Reduce how much old WAL we keep around. (#7098) 2020-04-07 15:55:57 +05:30
head_bench_test.go Make head Postings only return series in time range 2020-02-20 22:41:46 +01:00
head_test.go Fix typo (#7068) 2020-03-30 13:18:34 +05:30
isolation.go Do not attempt isolation for appendID == 0 2020-03-01 02:48:35 +01:00
mocks_test.go Revert head posting optimization 2020-03-25 20:19:33 +01:00
querier.go Merge pull request #7060 from prometheus/release-2.17 2020-03-27 15:57:07 +01:00
querier_bench_test.go Revert head posting optimization 2020-03-25 20:19:33 +01:00
querier_test.go Merge pull request #7060 from prometheus/release-2.17 2020-03-27 15:57:07 +01:00
README.md Edit TSDB README badges 2019-10-24 15:35:47 +05:30
repair.go tsdb: fix typo for wrong metric name (#6938) 2020-03-09 08:25:31 +00:00
repair_test.go Port tsdb to use pkg/labels. (#6326) 2019-11-18 11:53:33 -08:00
tsdbblockutil.go Fix golint issues. 2020-03-23 20:38:43 +01:00
wal.go Spelling (#6517) 2020-01-02 15:54:09 +01:00
wal_test.go Replace fileutil.ReadDir with ioutil.ReadDir (#7029) (#7033) 2020-04-06 19:04:20 +05:30

TSDB

GoDoc

This repository contains the Prometheus storage layer that is used in its 2.x releases.

A writeup of its design can be found here.

Based on the Gorilla TSDB white papers.

Video: Storing 16 Bytes at Scale from PromCon 2017.

See also the format documentation.