prometheus/tsdb
Oleg Zaytsev af0f6da5cb
Fix chunk overflow appending samples at a variable rate (#10607)
* Add a test with variable samples rate append

This test overflows the chunk created in memseries, and the total amount
of samples in the (only) mmapped chunk is 29, instead of the 65565
appended ones.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Cut new chunk when rate prediction was wrong

When appending samples at a slow rate, and then appending at a higher
rate, the prediction we made to cut a new chunk is no longer valid.
Sometimes this can even cause an overflow in the chunk, if more samples
than uint16 can hold are appended.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Improve comment on 2*samplesPerChunk

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Assert that all chunks have less than 240 samples

Also, trigger new chunk at 240, not at more than 240

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2022-04-20 14:54:20 +02:00
..
agent tsdb/agent: Ignore duplicate exemplars (#10595) 2022-04-18 11:41:04 -04:00
chunkenc Add SyncForState Implementation for Ruler HA (#10070) 2022-03-29 02:16:46 +02:00
chunks Disable chunk write queue by default, allow user to configure the exact size (#10425) 2022-03-11 17:26:59 +01:00
docs Update wal.md (#10442) 2022-03-15 22:33:45 +05:30
encoding Format Go source files using 'gofumpt -w -s -extra' 2021-11-02 19:52:34 +01:00
errors Address golint failures from revive 2021-10-23 00:53:11 +02:00
fileutil tsdb: Add windows arm64 support. 2021-11-09 11:07:27 +01:00
goversion Add new Go build tags. 2021-08-27 10:24:14 +02:00
index tsdb: more efficient sorting of postings read from WAL at startup (#10500) 2022-03-30 15:10:19 +05:30
record Add basic initial developer docs for TSDB (#9451) 2021-11-17 15:51:27 +05:30
test Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
testdata tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 2020-08-11 06:56:08 +01:00
tombstones refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
tsdbutil storage: fix bug #10027 in iterators' Seek method 2021-12-16 12:07:35 +01:00
wal Run gofumpt on all files (#10392) 2022-03-03 17:21:05 +01:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
block_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
blockwriter.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
blockwriter_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact.go CompactBlockMetas should produce correct mint/maxt for overlapping blocks. (#10108) 2022-01-05 15:10:00 +05:30
compact_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
db.go deleting *.tmp WAL files on startup (#10317) 2022-03-24 16:14:14 +05:30
db_test.go deleting *.tmp WAL files on startup (#10317) 2022-03-24 16:14:14 +05:30
example_test.go Fix Example() function in TSDB (#10153) 2022-01-11 17:24:03 +05:30
exemplar.go tsdb: use simpler map key to improve exemplar ingest performance (#10111) 2022-01-06 15:58:58 +05:30
exemplar_test.go tsdb: use simpler map key to improve exemplar ingest performance (#10111) 2022-01-06 15:58:58 +05:30
head.go Fix panic on query when m-map replay fails with snapshot enabled (#10348) 2022-02-25 08:53:40 -07:00
head_append.go Fix chunk overflow appending samples at a variable rate (#10607) 2022-04-20 14:54:20 +02:00
head_bench_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
head_read.go Label values with matchers by intersecting postings (#9907) 2021-12-28 15:59:03 +01:00
head_test.go Fix chunk overflow appending samples at a variable rate (#10607) 2022-04-20 14:54:20 +02:00
head_wal.go Fix data races in WAL replay (#10571) 2022-04-12 16:00:20 +05:30
isolation.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
isolation_test.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
mocks_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
querier.go Label values with matchers by intersecting postings (#9907) 2021-12-28 15:59:03 +01:00
querier_bench_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
querier_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
README.md Add basic initial developer docs for TSDB (#9451) 2021-11-17 15:51:27 +05:30
repair.go Switched to go-kit/log 2021-06-11 12:28:36 -04:00
repair_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
tsdbblockutil.go TSDB: demistify SeriesRefs and ChunkRefs (#9536) 2021-11-06 15:40:04 +05:30
wal.go fix potential goroutine leaks 2021-12-17 18:35:30 -05:00
wal_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Due to an issue with versioning, the "latest" docs shown on Godoc are outdated. Instead you may use the docs for v2.31.1

Documentation

External resources

A series of blog posts explaining different components of TSDB: