prometheus/tsdb
Chris Marchbanks 7c7dafc321
Do not snappy encode if record is too large (#8790)
Snappy cannot encode records larger than ~3.7 GB and will panic if an
encoding is attempted. Check to make sure that the record is smaller
than this before encoding.

In the future, we could improve this behavior to still compress large
records (or break them up into smaller records), but this avoids the
panic for users with very large single scrape targets.

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2021-05-06 12:56:45 -06:00
..
chunkenc Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
chunks Fixed TestChunkDiskMapper_WriteChunk_Chunk_IterateChunks for go1.16 (#8538) 2021-02-25 14:38:12 +05:30
docs/format Consistent naming and better names for fields in TSDB format docs (#8077) 2020-10-19 14:27:19 +05:30
encoding Avoid WriteAt for Postings. 2019-12-16 15:30:49 +00:00
errors MultiError: Refactored MultiError for more concise and safe usage. (#8066) 2020-10-28 15:24:58 +00:00
fileutil More info in mmap error message (#8058) 2020-10-15 12:58:13 +05:30
goversion tsdb: test for leaks (#7566) 2020-07-21 10:08:06 +02:00
index Add matchers to LabelValues() call (#8400) 2021-02-09 23:08:35 +05:30
record Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
test labels: Reduce allocated memory and latency of Hash method; Added tests. (#8025) 2020-10-15 11:31:28 +01:00
testdata tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 2020-08-11 06:56:08 +01:00
tombstones Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
tsdbutil Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
wal Do not snappy encode if record is too large (#8790) 2021-05-06 12:56:45 -06:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go Rolling tombstones clean up (#8007) 2021-02-17 11:02:43 +05:30
block_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
blockwriter.go add changes per comments, fix tests 2021-03-20 12:38:30 -07:00
blockwriter_test.go fix failing blockwriter tests 2021-03-01 06:16:05 -08:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact.go Add support for smaller block chunk segment allocations (#8478) 2021-04-15 14:25:01 +05:30
compact_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
db.go Add support for smaller block chunk segment allocations (#8478) 2021-04-15 14:25:01 +05:30
db_test.go Fix DB tests in the default branch 2021-02-18 23:56:27 +01:00
exemplar.go tsdb: move exemplar series labels to index entry (#8783) 2021-05-05 18:51:16 +01:00
exemplar_test.go tsdb: move exemplar series labels to index entry (#8783) 2021-05-05 18:51:16 +01:00
head.go Fix memSafeIterator.Seek() (#8748) 2021-04-27 00:43:22 +02:00
head_bench_test.go TSDB: optimize series creation on PreCreation() failure (#8620) 2021-03-18 20:53:50 +05:30
head_test.go Fix memSafeIterator.Seek() (#8748) 2021-04-27 00:43:22 +02:00
isolation.go Optimise lowWatermark in Isolation (#7332) 2020-06-03 20:09:05 +02:00
isolation_test.go Optimise lowWatermark in Isolation (#7332) 2020-06-03 20:09:05 +02:00
mocks_test.go Move away from testutil, refactor imports (#8087) 2020-10-22 11:00:08 +02:00
querier.go Correct spelling of "iterable" (#8713) 2021-04-12 21:43:42 +01:00
querier_bench_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
querier_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
README.md Link blog post on TSDB queries in tsdb/README.md (#8391) 2021-01-21 17:14:40 +05:30
repair.go MultiError: Refactored MultiError for more concise and safe usage. (#8066) 2020-10-28 15:24:58 +00:00
repair_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
tsdbblockutil.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
wal.go Update golangci-lint to 1.36.0 2021-02-04 08:53:33 -06:00
wal_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00

TSDB

GoDoc

This directory contains the Prometheus storage layer that is used in its 2.x releases.

A writeup of its design can be found here.

Based on the Gorilla TSDB white papers.

Video: Storing 16 Bytes at Scale from PromCon 2017.

See also the format documentation.

A series of blog posts explaining different components of TSDB: