mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-28 06:59:40 -08:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity
396 commits 243 branches 444 tags 649 MiB
Commit graph

143 commits

Author SHA1 Message Date
Fabian Reinartz 8c31c6e934 Make concurrent head chunk reads safe, fix misc races 
This adds a 4 sample buffer to every head chunk. The XOR
compression scheme may edit bytes in place. The minimum size
of a sample is 2 bits. So keeping the last 4 samples in an in-memory
buffer makes it safe to query the preceeding ones while samples
are added
 2017-01-09 16:51:39 +01:00
Fabian Reinartz 1943f8d1bb Fix head block stats races 2017-01-07 18:02:17 +01:00
Fabian Reinartz 6aa922c5a6 Fix races 2017-01-07 16:20:32 +01:00
Fabian Reinartz 300f4e2abf Use separate lock for series creation 
This uses the head block's own lock to only lock if new series were
encountered.
In the general append case we just need to hold a
 2017-01-06 18:10:50 +01:00
Fabian Reinartz 63e12807da Don't update head postings mapper on every append 2017-01-06 16:43:18 +01:00
Fabian Reinartz 71efd2e08d Periodically fsync WAL, make head cut async 2017-01-06 15:18:06 +01:00
Fabian Reinartz 96c2bd249f Handle compaction trigger and reinitializing in DB 2017-01-06 13:03:23 +01:00
Fabian Reinartz 937cdb579c Switch to sequential block names 
This changes block directory names from the int64 timestamp
to sequential numbering.
 2017-01-06 10:45:03 +01:00
Fabian Reinartz 9790aa98ac Add postings wrapper that emits head postings in label set order 
This adds a position mapper that takes series from a head block
in the order they were appended and creates a mapping representing
them in order of their label sets.

Write-repair of the postings list would cause very expensive writing.
Hence, we keep them as they are and only apply the postition mapping
at the very end, after a postings list has been sufficienctly reduced
through intersections etc.
 2017-01-05 16:05:42 +01:00
Fabian Reinartz 5aa7f7cce8 Compact head block into persisted block 2017-01-04 21:11:15 +01:00
Fabian Reinartz 3f72d5d027 Fix last timestamp initialization 
This initializes the chunkDesc's last timestamp to the minimum
value so initial samples with a timestamp of 0 (e.g. in tests)
are not accidentally dropped.
 2017-01-04 14:06:40 +01:00
Fabian Reinartz ac49f8c15e Consolidate persistence and compaction 2017-01-03 16:49:37 +01:00
Fabian Reinartz e7f04d14d5 Lock mmapped files 2017-01-03 10:09:20 +01:00
Fabian Reinartz 91b65b55e7 Run persistence in separate goroutine 2017-01-02 22:24:35 +01:00
Fabian Reinartz a648ef5252 Convert persister into function 2017-01-02 16:58:47 +01:00
Fabian Reinartz dbd2b21d2e Make persistence atomic 2017-01-02 14:41:13 +01:00
Fabian Reinartz beb842a856 Change block interface 
This changes the block interface to directly expose index
and series readers rather than the direct querier
 2017-01-02 11:12:28 +01:00
Fabian Reinartz 5d7ec06e04 Use ChunkMeta in SeriesWriter 2017-01-01 20:01:17 +01:00
Fabian Reinartz c00d17e691 Modify IndexReader API to accomodate compaction 
This changes the IndexReader API to expose plain labels
and chunk meta information instead of a Series interface.
Dropping of irrelevant chunks is moved into the querier.

A LabelIndices method is added to query for existing label
value indices.
 2016-12-31 16:24:04 +01:00
Fabian Reinartz 1e1a37b15b Remove double-reference in chunk hashmap 2016-12-31 10:19:02 +01:00
Fabian Reinartz 675f0886f0 Append to chunks cannot error 2016-12-31 10:10:27 +01:00
Fabian Reinartz 7280533c42 Add basic shard metrics 2016-12-31 09:48:49 +01:00
Fabian Reinartz 201d7687b6 Fix uniqueness of new series 2016-12-26 16:55:32 +01:00
Fabian Reinartz 787199a88e Fix erroneous value assignments 2016-12-22 20:57:00 +01:00
Fabian Reinartz 8aba95048a Rename to OpenHeadBlock 2016-12-22 20:00:24 +01:00
Fabian Reinartz 1dde3b6d31 Add WAL decoder+loading and benchmarks 2016-12-22 15:18:33 +01:00
Fabian Reinartz 0b8c77361e Add initial WAL writing 2016-12-22 12:05:24 +01:00
Fabian Reinartz 2a825f6c28 Consolidate mem index into HeadBlock 2016-12-22 01:12:28 +01:00
Fabian Reinartz ede733ab6c Extract labels package 2016-12-21 09:39:01 +01:00
Fabian Reinartz 1b23d62e3f Properly close files before reopening 2016-12-19 22:37:03 +01:00
Fabian Reinartz 282d9ae6e2 Implement label value queries in all layers. 2016-12-19 12:26:25 +01:00
Fabian Reinartz bad93d8d57 Extract head serialization into Head method 2016-12-18 14:43:27 +01:00
Fabian Reinartz b08f82fa4e Pre-select relevant chunks on series access. 
This adds interval metadata to indexed chunks. The queried interval
is used to filter chunks when queried from the index to save
unnecessary accesses of the chunks file.

This is especially relevant for series that come and go often and larger
files.
 2016-12-16 12:13:17 +01:00
Fabian Reinartz bd77103a49 Add stats serialization, load querier of all blocks 2016-12-15 16:14:33 +01:00
Fabian Reinartz d56b281006 Rename Iterator to Postings 2016-12-15 07:05:29 +01:00
Fabian Reinartz ca89080128 Misc fixes for initial Prometheus integration 2016-12-14 18:38:46 +01:00
Fabian Reinartz 14dbc59f2b cleanup and switching removal of unsafe calls. 2016-12-10 18:09:57 +01:00
Fabian Reinartz 40a451694f Refactor persistence into interfaces 2016-12-09 20:45:46 +01:00
Fabian Reinartz 74f8dfd95d Persist blocks periodically 2016-12-09 13:41:38 +01:00
Fabian Reinartz 0cf8bb9e53 Move sub-indexes into single index structure 2016-12-09 10:41:51 +01:00
Fabian Reinartz 8aa99a3ebd misc 2016-12-09 10:00:14 +01:00
Fabian Reinartz 2c34a15fe6 Add initial seriailization of block data 2016-12-08 17:43:10 +01:00
Fabian Reinartz 3ef7da33c8 Restructure files 2016-12-08 12:21:03 +01:00
Renamed from shard.go (Browse further)