mirror of
https://github.com/prometheus/prometheus.git
synced 2024-09-20 07:47:31 -07:00
0df3489275
* Write chunks via queue, predicting the refs Our load tests have shown that there is a latency spike in the remote write handler whenever the head chunks need to be written, because chunkDiskMapper.WriteChunk() blocks until the chunks are written to disk. This adds a queue to the chunk disk mapper which makes the WriteChunk() method non-blocking unless the queue is full. Reads can still be served from the queue. Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * address PR feeddback Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * initialize metrics without .Add(0) Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * change isRunningMtx to normal lock Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * do not re-initialize chunkrefmap Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * update metric outside of lock scope Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * add benchmark for adding job to chunk write queue Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * remove unnecessary "success" var Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * gofumpt -extra Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * avoid WithLabelValues call in addJob Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * format comments Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * addressing PR feedback Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * rename cutExpectRef to cutAndExpectRef Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * use head.Init() instead of .initTime() Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * address PR feedback Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * PR feedback Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * update test according to PR feedback Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * replace callbackWg -> awaitCb Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * better test of truncation with empty files Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * replace callbackWg -> awaitCb Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| agent | ||
| chunkenc | ||
| chunks | ||
| docs | ||
| encoding | ||
| errors | ||
| fileutil | ||
| goversion | ||
| index | ||
| record | ||
| test | ||
| testdata | ||
| tombstones | ||
| tsdbutil | ||
| wal | ||
| .gitignore | ||
| block.go | ||
| block_test.go | ||
| blockwriter.go | ||
| blockwriter_test.go | ||
| CHANGELOG.md | ||
| compact.go | ||
| compact_test.go | ||
| db.go | ||
| db_test.go | ||
| example_test.go | ||
| exemplar.go | ||
| exemplar_test.go | ||
| head.go | ||
| head_append.go | ||
| head_bench_test.go | ||
| head_read.go | ||
| head_test.go | ||
| head_wal.go | ||
| isolation.go | ||
| isolation_test.go | ||
| mocks_test.go | ||
| querier.go | ||
| querier_bench_test.go | ||
| querier_test.go | ||
| README.md | ||
| repair.go | ||
| repair_test.go | ||
| tsdbblockutil.go | ||
| wal.go | ||
| wal_test.go | ||
TSDB
This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.
Due to an issue with versioning, the "latest" docs shown on Godoc are outdated. Instead you may use the docs for v2.31.1
Documentation
External resources
- A writeup of the original design can be found here.
- Video: Storing 16 Bytes at Scale from PromCon 2017.
- Compression is based on the Gorilla TSDB white paper.
A series of blog posts explaining different components of TSDB: