prometheus/tsdb
Patryk Prus 61aa82865d
Some checks failed
buf.build / lint and publish (push) Has been cancelled
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
TSDB: keep duplicate series records in checkpoints while their samples may still be present (#16060)
Renames the head's deleted map to walExpiries, and creates entries for any
duplicate series records encountered during WAL replay, with the expiry set
to the highest current WAL segment number. Any subsequent WAL
checkpoints will see the duplicate series entry in the walExpiries map, and
keep the series record until the last WAL segment that could contain its
samples is deleted.

Other considerations:

WBL: series records aren't written to the WBL, so there are no duplicates to deal with
agent mode: has its own WAL replay logic that handles duplicate series records differently, and is outside the scope of this PR
2025-03-05 13:45:08 -05:00
..
agent TSDB: keep duplicate series records in checkpoints while their samples may still be present (#16060) 2025-03-05 13:45:08 -05:00
chunkenc tsdb/chunkenc: don't reuse custom value slices between histograms 2024-11-29 16:28:09 +11:00
chunks chore: enable unused-parameter from revive 2025-02-19 19:50:28 +01:00
docs Merge branch 'main' into cedwards/nhcb-wal-wbl 2025-01-02 12:50:19 +01:00
encoding Attempt for record type 2024-12-05 09:21:47 -08:00
errors Enable default revive rules (#13068) 2023-11-29 17:23:34 +00:00
fileutil chore: enable unused-parameter from revive 2025-02-19 19:50:28 +01:00
goversion remove obsolete build tag 2024-01-17 22:26:32 +08:00
index chore: enable unused-parameter from revive 2025-02-19 19:50:28 +01:00
record chore: enable unused-parameter from revive 2025-02-19 19:50:28 +01:00
testdata tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 2020-08-11 06:56:08 +01:00
tombstones chore!: adopt log/slog, remove go-kit/log 2024-10-07 15:58:50 -04:00
tsdbutil fix: os.MkdirTemp with t.TempDir (#15860) 2025-01-31 14:32:20 +00:00
wlog TSDB: keep duplicate series records in checkpoints while their samples may still be present (#16060) 2025-03-05 13:45:08 -05:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go Upgrade to github.com/oklog/ulid/v2 (#16168) 2025-03-05 16:03:25 +01:00
block_test.go chore: enable unused-parameter from revive 2025-02-19 19:50:28 +01:00
blockwriter.go Upgrade to github.com/oklog/ulid/v2 (#16168) 2025-03-05 16:03:25 +01:00
blockwriter_test.go feat: Allow customizing TSDB postings decoder (#13567) 2024-11-11 07:59:24 +01:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact.go Upgrade to github.com/oklog/ulid/v2 (#16168) 2025-03-05 16:03:25 +01:00
compact_test.go Upgrade to github.com/oklog/ulid/v2 (#16168) 2025-03-05 16:03:25 +01:00
db.go Upgrade to github.com/oklog/ulid/v2 (#16168) 2025-03-05 16:03:25 +01:00
db_test.go TSDB: keep duplicate series records in checkpoints while their samples may still be present (#16060) 2025-03-05 13:45:08 -05:00
example_test.go Add context argument to Querier.Select (#12660) 2023-09-12 12:37:38 +02:00
exemplar.go tsdb.CircularExemplarStorage: Avoid racing (#15231) 2024-10-29 10:40:46 +01:00
exemplar_test.go tsdb.CircularExemplarStorage: Avoid racing (#15231) 2024-10-29 10:40:46 +01:00
head.go TSDB: keep duplicate series records in checkpoints while their samples may still be present (#16060) 2025-03-05 13:45:08 -05:00
head_append.go Use new record type only for NHCB 2024-12-06 13:46:20 -08:00
head_bench_test.go Revert "Fix MemPostings.Add and MemPostings.Get data race (#15141)" 2024-11-03 12:30:34 +00:00
head_dedupelabels.go chore!: adopt log/slog, remove go-kit/log 2024-10-07 15:58:50 -04:00
head_other.go chore: enable unused-parameter from revive 2025-02-19 19:50:28 +01:00
head_read.go chore: enable unused-parameter from revive 2025-02-19 19:50:28 +01:00
head_read_test.go TSDB: Simplify OOO Select by copying the head chunk (#14396) 2024-07-03 15:08:07 +01:00
head_test.go TSDB: keep duplicate series records in checkpoints while their samples may still be present (#16060) 2025-03-05 13:45:08 -05:00
head_wal.go TSDB: keep duplicate series records in checkpoints while their samples may still be present (#16060) 2025-03-05 13:45:08 -05:00
isolation.go chore: enable early-return from revive 2025-02-10 22:08:43 +01:00
isolation_test.go chore: enable unused-parameter from revive 2025-02-19 19:50:28 +01:00
mocks_test.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
ooo_head.go TSDB: Remove code for querying OOO-head only 2024-08-14 13:41:13 +01:00
ooo_head_read.go Upgrade to github.com/oklog/ulid/v2 (#16168) 2025-03-05 16:03:25 +01:00
ooo_head_read_test.go Rename old histogram record type, use old names for new records 2024-12-05 09:21:47 -08:00
ooo_head_test.go fix TestOOOHeadChunkReader_Chunk on 32-bit 2024-12-16 10:45:07 -05:00
ooo_isolation.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
ooo_isolation_test.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
querier.go Upgrade to github.com/oklog/ulid/v2 (#16168) 2025-03-05 16:03:25 +01:00
querier_bench_test.go TSDB benchmarks: Commit periodically to speed up init 2024-12-30 17:42:56 +00:00
querier_test.go Upgrade to github.com/oklog/ulid/v2 (#16168) 2025-03-05 16:03:25 +01:00
README.md Fixed broken link in tsdb README.md 2022-10-07 16:20:20 +00:00
repair.go chore!: adopt log/slog, remove go-kit/log 2024-10-07 15:58:50 -04:00
repair_test.go feat: Allow customizing TSDB postings decoder (#13567) 2024-11-11 07:59:24 +01:00
testutil.go Merge branch 'main' into cedwards/nhcb-wal-wbl 2025-01-02 12:50:19 +01:00
tsdbblockutil.go enable errorf rule from perfsprint linter 2024-11-06 16:50:36 +01:00

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Documentation

External resources

A series of blog posts explaining different components of TSDB: