prometheus/tsdb
Marco Pracucci 501bc6419e
Add ShardedPostings() support to TSDB (#10421)
This PR is a reference implementation of the proposal described in #10420.

In addition to what described in #10420, in this PR I've introduced labels.StableHash(). The idea is to offer an hashing function which doesn't change over time, and that's used by query sharding in order to get a stable behaviour over time. The implementation of labels.StableHash() is the hashing function used by Prometheus before stringlabels, and what's used by Grafana Mimir for query sharding (because built before stringlabels was a thing).

Follow up work
As mentioned in #10420, if this PR is accepted I'm also open to upload another foundamental piece used by Grafana Mimir query sharding to accelerate the query execution: an optional, configurable and fast in-memory cache for the series hashes.

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2024-01-29 11:57:27 +00:00
..
agent agent: add BenchmarkCreateSeries 2023-12-31 10:23:43 +00:00
chunkenc Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
chunks Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
docs
encoding tsdb/encoding: use Go standard errors package 2023-11-11 19:01:11 +01:00
errors Enable default revive rules (#13068) 2023-11-29 17:23:34 +00:00
fileutil remove obsolete build tag 2024-01-17 22:26:32 +08:00
goversion remove obsolete build tag 2024-01-17 22:26:32 +08:00
index Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
record Move metric type definitions to common/model 2023-12-19 18:56:54 +00:00
testdata
tombstones Update tombstones.go 2023-11-11 19:22:06 +01:00
tsdbutil Revert "Adding small test update for temp dir using t.TempDir (#13293)" 2023-12-30 19:17:30 +00:00
wlog tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
.gitignore
block.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
block_test.go TestHeadLabelValuesWithMatchers: Add test case (#13414) 2024-01-24 10:47:56 +01:00
blockwriter.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
blockwriter_test.go
CHANGELOG.md
compact.go Fix regressions introduced by #13242 2024-01-16 12:00:53 +01:00
compact_test.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
db.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
db_test.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
example_test.go
exemplar.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
exemplar_test.go golangci-lint: enable testifylint linter (#13254) 2023-12-07 11:35:01 +00:00
head.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
head_append.go Move metric type definitions to common/model 2023-12-19 18:56:54 +00:00
head_bench_test.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
head_read.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
head_read_test.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
head_test.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
head_wal.go Move metric type definitions to common/model 2023-12-19 18:56:54 +00:00
isolation.go
isolation_test.go
mocks_test.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
ooo_head.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
ooo_head_read.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
ooo_head_read_test.go Optimize label values with matchers by taking shortcuts (#13426) 2024-01-23 11:40:21 +01:00
ooo_head_test.go ci(lint): enable godot; append dot at the end of comments 2023-10-31 19:53:38 +02:00
ooo_isolation.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
ooo_isolation_test.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
querier.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
querier_bench_test.go Optimize label values with matchers by taking shortcuts (#13426) 2024-01-23 11:40:21 +01:00
querier_test.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
README.md
repair.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
repair_test.go
tsdbblockutil.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
wal.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
wal_test.go remove obsolete build tag 2024-01-17 22:26:32 +08:00

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Documentation

External resources

A series of blog posts explaining different components of TSDB: