prometheus/tsdb
György Krajcsovits 41c076196e New cases in Test_ChunkQuerier_OOOQuery and Test_Querier_OOOQuery
Case 1: OOO in-memory head chunk overlaps with first mmaped in-order chunk.

Query: |----------------------------------------------------------------|
InO:    |------mmap---------------||---------mem----------------------|
OOO:     |-----mem-----------|

This triggers ChunkOrIterableWithCopy not including OOO head chunks bug.

Similar to #14693 however testing the end of the interval doesn't
trigger the problem because there the in-order head chunk will be
trimmed with a tombstone, causing the code to switch to ChunkOrIterable
which was fixed.
See a36d1a8a92/tsdb/querier.go (L646)
where len(p.bufIter.Intervals) will be non zero, because it includes the
tombstone to trim the result to the query max time.

Thus a new test is added to check the overlap at the beginning of the
interval that has a separate chunk, which does not need trimming.

Note: same test doesn't fail for sample querier in Test_Querier_OOOQuery
as that doesn't use copy, that is copyHeadChunk is false in the if
condition above.

Case 2:

OOO mmaped head chunk overlaps with first mmaped in-order chunk.

Query: |----------------------------------------------------------------|
InO:    |------mmap---------------||---------mem----------------------|
OOO:     |-----mmap-----------|                             |--mem--|

In this case the meta contains the reference of the in-order chunk and
no indication that a merge is needed with the OOO mmaped chunk.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-23 15:50:47 +02:00
..
agent [PRW 2.0] Merging remote-write-2.0 feature branch to main (PRW 2.0 support + metadata in WAL) (#14395) 2024-07-04 14:29:20 -07:00
chunkenc Fix corrupting spans via iterator sharing 2024-08-06 16:51:20 +02:00
chunks lint: Revamp our linting rules, mostly around doc comments 2024-08-22 17:36:11 +02:00
docs Fix language in docs and comments (#14041) 2024-05-08 17:57:09 +02:00
encoding lint: Revamp our linting rules, mostly around doc comments 2024-08-22 17:36:11 +02:00
errors Enable default revive rules (#13068) 2023-11-29 17:23:34 +00:00
fileutil tests: remove err from message when testify prints it already 2024-02-01 14:18:01 +00:00
goversion remove obsolete build tag 2024-01-17 22:26:32 +08:00
index lint: Revamp our linting rules, mostly around doc comments 2024-08-22 17:36:11 +02:00
record Fix a couple of comments 2024-07-05 15:25:42 +02:00
testdata tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 2020-08-11 06:56:08 +01:00
tombstones Update tombstones.go 2023-11-11 19:22:06 +01:00
tsdbutil Ignore stale histograms for counter reset detection 2024-07-26 10:08:31 +02:00
wlog lint: Revamp our linting rules, mostly around doc comments 2024-08-22 17:36:11 +02:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go TSDB: Review feedback 2024-08-14 13:41:44 +01:00
block_test.go tsdb: Extend compactor interface to allow compactions to create multiple output blocks (#14143) 2024-06-12 17:31:25 -04:00
blockwriter.go tsdb: Extend compactor interface to allow compactions to create multiple output blocks (#14143) 2024-06-12 17:31:25 -04:00
blockwriter_test.go Add a chunk size limit in bytes (#12054) 2023-08-24 15:21:17 +02:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact.go TSDB: Change block populator to accept postings index function (#14213) 2024-06-25 09:21:48 +01:00
compact_test.go feat: allow to delay head compaction start time helping Prometheus instances to 2024-08-07 17:10:27 +02:00
db.go lint: Revamp our linting rules, mostly around doc comments 2024-08-22 17:36:11 +02:00
db_test.go New cases in Test_ChunkQuerier_OOOQuery and Test_Querier_OOOQuery 2024-08-23 15:50:47 +02:00
example_test.go Add context argument to Querier.Select (#12660) 2023-09-12 12:37:38 +02:00
exemplar.go [ENHANCEMENT] TSDB: Save map lookup on validation 2024-05-30 09:17:11 +01:00
exemplar_test.go [Test] TSDB: BenchmarkResizeExemplar multiple per series 2024-05-30 09:17:11 +01:00
head.go Revert the option regardless of error 2024-07-30 11:31:31 +02:00
head_append.go Upgrade golangci-lint to v1.60.1 2024-08-18 12:13:25 +02:00
head_bench_test.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
head_dedupelabels.go tsdb: reset symbol table for exemplars periodically 2024-07-05 10:19:07 +01:00
head_other.go lint: Revamp our linting rules, mostly around doc comments 2024-08-22 17:36:11 +02:00
head_read.go [BUGFIX] TSDB: Fix query overlapping in-order and ooo head (#14693) 2024-08-21 14:24:20 +01:00
head_read_test.go TSDB: Simplify OOO Select by copying the head chunk (#14396) 2024-07-03 15:08:07 +01:00
head_test.go Upgrade golangci-lint to v1.60.1 2024-08-18 12:13:25 +02:00
head_wal.go TSDB: Never go over maximum number of OOO chunks 2024-08-14 13:41:59 +01:00
isolation.go tsdb: create isolation transaction slice on demand 2023-10-21 13:45:47 +00:00
isolation_test.go tsdb: turn off transaction isolation for head compaction (#11317) 2022-09-27 19:31:23 +05:30
mocks_test.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
ooo_head.go TSDB: Remove code for querying OOO-head only 2024-08-14 13:41:13 +01:00
ooo_head_read.go lint: Revamp our linting rules, mostly around doc comments 2024-08-22 17:36:11 +02:00
ooo_head_read_test.go TSDB: Remove OOOHeadIndexReader 2024-08-14 13:41:13 +01:00
ooo_head_test.go Fix ToEncodedChunks minT for recoded chunks 2024-08-08 15:22:46 +02:00
ooo_isolation.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
ooo_isolation_test.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
querier.go tsdb: Fix ValNone typo in comment 2024-08-23 08:20:20 +02:00
querier_bench_test.go TSDB: Review feedback 2024-08-14 13:41:44 +01:00
querier_test.go TSDB tests: Fix up BenchmarkQueries 2024-08-14 13:41:04 +01:00
README.md Fixed broken link in tsdb README.md 2022-10-07 16:20:20 +00:00
repair.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
repair_test.go tsdb tests: use go-cmp instead of DeepEquals 2024-02-08 19:32:33 +00:00
testutil.go Testutil refactoring 2024-07-03 09:28:38 -07:00
tsdbblockutil.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Documentation

External resources

A series of blog posts explaining different components of TSDB: