prometheus/tsdb
nberkley f9e2dd0697
Add support for smaller block chunk segment allocations (#8478)
* Add support for --storage.tsdb.max-chunk-size to suport small chunks for space limited prometheus instances.

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update tsdb/compact.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update tsdb/db.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update cmd/prometheus/main.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Change naming scheme to

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Add a lower bound to --storage.tsdb.max-block-chunk-segment-size

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update storage.md to explain what a chunk segment is

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Apply suggestions from code review

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Force tests

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Fix code style

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-04-15 14:25:01 +05:30
..
chunkenc Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
chunks Fixed TestChunkDiskMapper_WriteChunk_Chunk_IterateChunks for go1.16 (#8538) 2021-02-25 14:38:12 +05:30
docs/format Consistent naming and better names for fields in TSDB format docs (#8077) 2020-10-19 14:27:19 +05:30
encoding Avoid WriteAt for Postings. 2019-12-16 15:30:49 +00:00
errors MultiError: Refactored MultiError for more concise and safe usage. (#8066) 2020-10-28 15:24:58 +00:00
fileutil More info in mmap error message (#8058) 2020-10-15 12:58:13 +05:30
goversion tsdb: test for leaks (#7566) 2020-07-21 10:08:06 +02:00
index Add matchers to LabelValues() call (#8400) 2021-02-09 23:08:35 +05:30
record Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
test labels: Reduce allocated memory and latency of Hash method; Added tests. (#8025) 2020-10-15 11:31:28 +01:00
testdata tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 2020-08-11 06:56:08 +01:00
tombstones Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
tsdbutil Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
wal Fix golint issue caused by typo 2021-04-12 09:57:41 +08:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go Rolling tombstones clean up (#8007) 2021-02-17 11:02:43 +05:30
block_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
blockwriter.go add changes per comments, fix tests 2021-03-20 12:38:30 -07:00
blockwriter_test.go fix failing blockwriter tests 2021-03-01 06:16:05 -08:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact.go Add support for smaller block chunk segment allocations (#8478) 2021-04-15 14:25:01 +05:30
compact_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
db.go Add support for smaller block chunk segment allocations (#8478) 2021-04-15 14:25:01 +05:30
db_test.go Fix DB tests in the default branch 2021-02-18 23:56:27 +01:00
exemplar.go Add circular in-memory exemplars storage (#6635) 2021-03-16 15:17:45 +05:30
exemplar_test.go Add circular in-memory exemplars storage (#6635) 2021-03-16 15:17:45 +05:30
head.go GetRef() now returns the label set (#8641) 2021-03-24 15:24:58 +00:00
head_bench_test.go TSDB: optimize series creation on PreCreation() failure (#8620) 2021-03-18 20:53:50 +05:30
head_test.go Add circular in-memory exemplars storage (#6635) 2021-03-16 15:17:45 +05:30
isolation.go Optimise lowWatermark in Isolation (#7332) 2020-06-03 20:09:05 +02:00
isolation_test.go Optimise lowWatermark in Isolation (#7332) 2020-06-03 20:09:05 +02:00
mocks_test.go Move away from testutil, refactor imports (#8087) 2020-10-22 11:00:08 +02:00
querier.go Correct spelling of "iterable" (#8713) 2021-04-12 21:43:42 +01:00
querier_bench_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
querier_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
README.md Link blog post on TSDB queries in tsdb/README.md (#8391) 2021-01-21 17:14:40 +05:30
repair.go MultiError: Refactored MultiError for more concise and safe usage. (#8066) 2020-10-28 15:24:58 +00:00
repair_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
tsdbblockutil.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
wal.go Update golangci-lint to 1.36.0 2021-02-04 08:53:33 -06:00
wal_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00

TSDB

GoDoc

This directory contains the Prometheus storage layer that is used in its 2.x releases.

A writeup of its design can be found here.

Based on the Gorilla TSDB white papers.

Video: Storing 16 Bytes at Scale from PromCon 2017.

See also the format documentation.

A series of blog posts explaining different components of TSDB: