mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-23 12:44:05 -08:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity
prometheus/tsdb
History
Peter Štibraný ffc60d8397
Reduce chunk write queue memory usage 2 (#10874) 
* Job queue

This PR reimplements chan chunkWriteJob with custom buffered queue that should use less memory, because it doesn't preallocate entire buffer for maximum queue size at once. Instead it allocates individual "segments" with smaller size.

As elements are added to the queue, they fill individual segments. When elements are removed from the queue (and segments), empty segments can be thrown away. This doesn't change memory usage of the queue when it's full, but should decrease its memory footprint when it's empty (queue will keep max 1 segment in such case).

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>

* Modify test to work with low resolution timer.

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>

* Improve comments.

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2022-06-29 17:51:27 +05:30
..
agent unit tests: make all Labels sorted alphabetically (#10532) 2022-05-04 23:41:36 +02:00
chunkenc Add SyncForState Implementation for Ruler HA (#10070) 2022-03-29 02:16:46 +02:00
chunks Reduce chunk write queue memory usage 2 (#10874) 2022-06-29 17:51:27 +05:30
docs Update godoc links and remove note about TSDB versioning (#10754) 2022-05-26 18:34:43 +10:00
encoding Format Go source files using 'gofumpt -w -s -extra' 2021-11-02 19:52:34 +01:00
errors Address golint failures from revive 2021-10-23 00:53:11 +02:00
fileutil Make necessary change to compile promql parser to wasm (#10683) 2022-05-12 09:12:05 +02:00
goversion Add new Go build tags. 2021-08-27 10:24:14 +02:00
index Fix typo in symbol table size exceeded error message (#10746) 2022-05-25 10:40:36 +02:00
record Add basic initial developer docs for TSDB (#9451) 2021-11-17 15:51:27 +05:30
test Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
testdata tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 2020-08-11 06:56:08 +01:00
tombstones refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
tsdbutil refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
wal Remote Write: Rename confusing walDir parameter to dir (#10464) 2022-05-30 21:45:30 -07:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go fix typo (#10696) 2022-05-25 18:01:45 +02:00
block_test.go unit tests: make all Labels sorted alphabetically (#10532) 2022-05-04 23:41:36 +02:00
blockwriter.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
blockwriter_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact.go CompactBlockMetas should produce correct mint/maxt for overlapping blocks. (#10108) 2022-01-05 15:10:00 +05:30
compact_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
db.go Don't increment prometheus_tsdb_compactions_failed_total when context is canceled (#10772) 2022-06-17 11:21:43 +05:30
db_test.go test(tsdb): break up repeated test to avoid timeout (#10671) 2022-05-09 00:39:26 +02:00
example_test.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
exemplar.go tsdb: use simpler map key to improve exemplar ingest performance (#10111) 2022-01-06 15:58:58 +05:30
exemplar_test.go unit tests: make all Labels sorted alphabetically (#10532) 2022-05-04 23:41:36 +02:00
head.go fix(tsdb): inc mmap corruption counter on mmap out of sequence error (#10406) 2022-06-22 14:03:12 +05:30
head_append.go Fix chunk overflow appending samples at a variable rate (#10607) 2022-04-20 14:54:20 +02:00
head_bench_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
head_read.go Label values with matchers by intersecting postings (#9907) 2021-12-28 15:59:03 +01:00
head_test.go unit tests: make all Labels sorted alphabetically (#10532) 2022-05-04 23:41:36 +02:00
head_wal.go tsdb: reduce sleep time when reading WAL (#10859) 2022-06-12 11:54:11 +05:30
isolation.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
isolation_test.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
mocks_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
querier.go Label values with matchers by intersecting postings (#9907) 2021-12-28 15:59:03 +01:00
querier_bench_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
querier_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
README.md Update godoc links and remove note about TSDB versioning (#10754) 2022-05-26 18:34:43 +10:00
repair.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
repair_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
tsdbblockutil.go tsdb: commit data periodically in CreateBlock (#10788) 2022-06-17 11:26:19 +05:30
wal.go fix potential goroutine leaks 2021-12-17 18:35:30 -05:00
wal_test.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00

README.md

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Documentation

External resources

A series of blog posts explaining different components of TSDB: