prometheus/tsdb
Oleg Zaytsev 8553a98267
Optimize postings offset table reading (#11535)
* Add BenchmarkOpenBlock

* Use specific types when reading offset table

Instead of reading a generic-ish []string, we can read a generic type
which would be specifically labels.Label.

This avoid allocating a slice that escapes to the heap, making it both
faster and more efficient in terms of memory management.

* Update error message for unexpected number of keys

* s/posting offset table/postings offset table/

* Remove useless lastKey assignment

* Use two []bytes vars, simplify

Applied PR feedback: removed generics, moved the label indices reading
to that specific test as we're not using it in production anyway, we're
just testing what we've just built.

Also using two []bytes variables for name and value that use the backing
buffer instead of using strings, this reduces allocations a lot as we
only copy them when we store them (this is optimized by the compiler).

* Fix the dumb bug

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Marco Pracucci <marco@pracucci.com>
2022-11-14 17:48:16 +01:00
..
agent Merge remote-tracking branch 'upstream/main' into fix-conflict 2022-10-12 14:20:02 +05:30
chunkenc chunkenc: Slightly optimize xorWrite/xoRead (#11476) 2022-10-20 15:08:01 +05:30
chunks Replace sort.Strings and sort.Ints with faster slices.Sort (#11318) 2022-09-30 20:03:56 +05:30
docs Merge branch 'main' into sparsehistogram 2022-08-10 17:54:37 +02:00
encoding Merge 'main' into sparsehistogram 2022-10-05 22:14:49 +02:00
errors Address golint failures from revive 2021-10-23 00:53:11 +02:00
fileutil Update go to 1.19, set min version to 1.18 (#11279) 2022-09-07 11:30:48 +02:00
goversion Add new Go build tags. 2021-08-27 10:24:14 +02:00
index Optimize postings offset table reading (#11535) 2022-11-14 17:48:16 +01:00
record Merge 'main' into sparsehistogram 2022-10-05 22:14:49 +02:00
test Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
testdata tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 2020-08-11 06:56:08 +01:00
tombstones refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
tsdbutil Fix merge conflicts 2022-10-11 22:53:37 +05:30
wlog Merge remote-tracking branch 'upstream/main' into fix-conflict 2022-10-12 14:20:02 +05:30
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go tsdb: fix up sort call with faster slices.Sort (#11380) 2022-10-01 12:55:40 -04:00
block_test.go Optimize postings offset table reading (#11535) 2022-11-14 17:48:16 +01:00
blockwriter.go Merge 'main' into sparsehistogram 2022-10-05 22:14:49 +02:00
blockwriter_test.go tsdb: in tests use labels.FromStrings 2022-09-09 13:34:49 +02:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact.go tsdb: Fix chunk handling during appendHistogram 2022-07-06 18:44:53 +02:00
compact_test.go Fix merge conflicts 2022-10-11 22:53:37 +05:30
db.go Merge pull request #11447 from prometheus/sparsehistogram 2022-10-26 17:10:46 -04:00
db_test.go Merge remote-tracking branch 'upstream/main' into fix-conflict 2022-10-12 14:20:02 +05:30
example_test.go Merge branch 'main' into sparsehistogram 2022-05-04 13:37:13 +02:00
exemplar.go tsdb: use simpler map key to improve exemplar ingest performance (#10111) 2022-01-06 15:58:58 +05:30
exemplar_test.go tsdb: in tests use labels.FromStrings 2022-09-09 13:34:49 +02:00
head.go Fix alignment of atomic int64 (#11547) 2022-11-09 11:18:49 +01:00
head_append.go Fix merge conflicts 2022-10-11 22:53:37 +05:30
head_bench_test.go Add out-of-order sample support to the TSDB (#11075) 2022-09-20 22:35:50 +05:30
head_read.go Fix merge conflicts 2022-10-11 22:53:37 +05:30
head_read_test.go Fix merge conflicts 2022-10-11 22:53:37 +05:30
head_test.go Fix the build after conflict resolution 2022-10-12 17:59:42 +05:30
head_wal.go Fix the build after conflict resolution 2022-10-12 17:59:42 +05:30
isolation.go tsdb: turn off transaction isolation for head compaction (#11317) 2022-09-27 19:31:23 +05:30
isolation_test.go tsdb: turn off transaction isolation for head compaction (#11317) 2022-09-27 19:31:23 +05:30
mocks_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
ooo_head.go Fix merge conflicts 2022-10-11 22:53:37 +05:30
ooo_head_read.go Add out-of-order sample support to the TSDB (#11075) 2022-09-20 22:35:50 +05:30
ooo_head_read_test.go Fix merge conflicts 2022-10-11 22:53:37 +05:30
ooo_head_test.go Fix merge conflicts 2022-10-11 22:53:37 +05:30
querier.go Merge 'main' into sparsehistogram 2022-10-05 22:14:49 +02:00
querier_bench_test.go Add out-of-order sample support to the TSDB (#11075) 2022-09-20 22:35:50 +05:30
querier_test.go Merge 'main' into sparsehistogram 2022-10-05 22:14:49 +02:00
README.md Fixed broken link in tsdb README.md 2022-10-07 16:20:20 +00:00
repair.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
repair_test.go tsdb: in tests use labels.FromStrings 2022-09-09 13:34:49 +02:00
tsdbblockutil.go Add unit test for histogram append and various querying scenarios (#11194) 2022-08-29 15:35:03 +05:30
wal.go TSDB: Rename wal package to wlog (#11352) 2022-10-10 20:38:46 +05:30
wal_test.go TSDB: Rename wal package to wlog (#11352) 2022-10-10 20:38:46 +05:30

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Documentation

External resources

A series of blog posts explaining different components of TSDB: