Commit graph

614 commits

Author SHA1 Message Date
Fabian Reinartz 3f72d5d027 Fix last timestamp initialization
This initializes the chunkDesc's last timestamp to the minimum
value so initial samples with a timestamp of 0 (e.g. in tests)
are not accidentally dropped.
2017-01-04 14:06:40 +01:00
Fabian Reinartz 40cf215fba Hotfix missing sorting of head block series index 2017-01-04 09:47:20 +01:00
Fabian Reinartz ec99f99d3d Fix and test bug in shardSeriesSet 2017-01-03 19:02:42 +01:00
Fabian Reinartz ac49f8c15e Consolidate persistence and compaction 2017-01-03 16:49:37 +01:00
Fabian Reinartz e7f04d14d5 Lock mmapped files 2017-01-03 10:09:20 +01:00
Fabian Reinartz 91b65b55e7 Run persistence in separate goroutine 2017-01-02 22:24:35 +01:00
Fabian Reinartz a648ef5252 Convert persister into function 2017-01-02 16:58:47 +01:00
Fabian Reinartz dbd2b21d2e Make persistence atomic 2017-01-02 14:41:13 +01:00
Fabian Reinartz 62b8ded0a5 Rename iterator value getters to At() 2017-01-02 13:33:06 +01:00
Fabian Reinartz 5c45a1cc6f Remove shardSeriesSet caching 2017-01-02 12:05:52 +01:00
Fabian Reinartz beb842a856 Change block interface
This changes the block interface to directly expose index
and series readers rather than the direct querier
2017-01-02 11:12:28 +01:00
Fabian Reinartz 5d75a3dc7b Add basic compaction
This adds a basic compactor that will merge two persisted blocks into
one. It simply fully rewrites the index and concatenates the chunk
lists.

It just writes into the current working dir and doesn't properly handle
which blocks to compact for now.
2017-01-02 10:34:55 +01:00
Fabian Reinartz 5d7ec06e04 Use ChunkMeta in SeriesWriter 2017-01-01 20:01:17 +01:00
Fabian Reinartz c00d17e691 Modify IndexReader API to accomodate compaction
This changes the IndexReader API to expose plain labels
and chunk meta information instead of a Series interface.
Dropping of irrelevant chunks is moved into the querier.

A LabelIndices method is added to query for existing label
value indices.
2016-12-31 16:24:04 +01:00
Fabian Reinartz 1e1a37b15b Remove double-reference in chunk hashmap 2016-12-31 10:19:02 +01:00
Fabian Reinartz 675f0886f0 Append to chunks cannot error 2016-12-31 10:10:27 +01:00
Fabian Reinartz 7280533c42 Add basic shard metrics 2016-12-31 09:48:49 +01:00
Fabian Reinartz a009247ab7 Add matching of empty label 2016-12-30 19:36:28 +01:00
Fabian Reinartz eb4f366988 Add return error to Appender.Add 2016-12-29 11:03:39 +01:00
Fabian Reinartz 3a5468f251 Lock shards during querying and shutdown 2016-12-28 11:41:44 +01:00
Fabian Reinartz 286293802b Add merge postings 2016-12-28 11:02:19 +01:00
Fabian Reinartz ab7fbc05ad Zero timestamp as base, use binary search list postings 2016-12-28 08:50:20 +01:00
Fabian Reinartz dd72b52098 Add postings intersection 2016-12-27 11:32:10 +01:00
Fabian Reinartz f8111cef0e Fix chunk series iterator seeking 2016-12-26 16:55:44 +01:00
Fabian Reinartz 201d7687b6 Fix uniqueness of new series 2016-12-26 16:55:32 +01:00
Fabian Reinartz 5efe1d178e labels: add Compare and String methods 2016-12-24 13:49:35 +01:00
Fabian Reinartz 0d0c5cfaf1 labels: add string constructor, expose matcher 2016-12-24 10:19:46 +01:00
Fabian Reinartz 787199a88e Fix erroneous value assignments 2016-12-22 20:57:00 +01:00
Fabian Reinartz 8aba95048a Rename to OpenHeadBlock 2016-12-22 20:00:24 +01:00
Fabian Reinartz cabd7e4ebc Add small testdata file 2016-12-22 19:15:37 +01:00
Fabian Reinartz e7edae39b2 Use page buffered writer for WAL 2016-12-22 16:14:34 +01:00
Fabian Reinartz 9c6a72aadd Load head with WALs correctly 2016-12-22 15:54:39 +01:00
Fabian Reinartz 1dde3b6d31 Add WAL decoder+loading and benchmarks 2016-12-22 15:18:33 +01:00
Fabian Reinartz 0b8c77361e Add initial WAL writing 2016-12-22 12:05:24 +01:00
Fabian Reinartz 2a825f6c28 Consolidate mem index into HeadBlock 2016-12-22 01:12:28 +01:00
Fabian Reinartz 869cccf080 Test and fixes for buffered iterator 2016-12-21 16:06:33 +01:00
Fabian Reinartz 0a94f58f1a Fix test import of labels, simplify constructor names 2016-12-21 15:12:26 +01:00
Fabian Reinartz da2beb3e6d Fix zero division, add buffer series iterator 2016-12-21 13:04:51 +01:00
Fabian Reinartz dbca3453fb Add label clone benchmark 2016-12-21 10:37:38 +01:00
Fabian Reinartz ede733ab6c Extract labels package 2016-12-21 09:39:01 +01:00
Fabian Reinartz ee217adc7e Redfine append interface, remove old Prometheus storage from bench 2016-12-21 00:02:37 +01:00
Fabian Reinartz cddc29fa17 Fix labels comparison, fetch correct labels 2016-12-20 14:54:52 +01:00
Fabian Reinartz ce7f4106c2 Reda correct label number, fix buffered iterator panic 2016-12-20 14:21:50 +01:00
Fabian Reinartz d9ca4b47f5 Fix offset errors, fix persisted postings order 2016-12-20 13:14:55 +01:00
Fabian Reinartz 1b23d62e3f Properly close files before reopening 2016-12-19 22:37:03 +01:00
Fabian Reinartz 00a503129b Use contextualized and traced errors in reader 2016-12-19 22:29:49 +01:00
Fabian Reinartz 282d9ae6e2 Implement label value queries in all layers. 2016-12-19 12:26:25 +01:00
Fabian Reinartz aabb21f4b9 Add shard series set test 2016-12-19 11:44:11 +01:00
Fabian Reinartz bad93d8d57 Extract head serialization into Head method 2016-12-18 14:43:27 +01:00
Fabian Reinartz b08f82fa4e Pre-select relevant chunks on series access.
This adds interval metadata to indexed chunks. The queried interval
is used to filter chunks when queried from the index to save
unnecessary accesses of the chunks file.

This is especially relevant for series that come and go often and larger
files.
2016-12-16 12:13:17 +01:00