mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-27 14:39:40 -08:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity
162 commits 243 branches 444 tags 649 MiB
Commit graph

67 commits

Author SHA1 Message Date
Fabian Reinartz c20cc44b06 Add docs, write sequence number to meta.json 2017-01-29 08:11:47 +01:00
Fabian Reinartz 472c618c39 Drop out-of-bound samples 2017-01-19 15:03:57 +01:00
Fabian Reinartz d4779b374c Properly track and write meta file 2017-01-19 14:01:38 +01:00
Fabian Reinartz 9ddbd64d00 Move stats into meta.json file, cleanup, docs 2017-01-19 11:22:47 +01:00
Fabian Reinartz 5ceca3c810 Write to WAL before appending to memory storage 2017-01-17 16:33:58 +01:00
Fabian Reinartz 343dd9d94c Fix wrong byte size in WAL base ref 2017-01-17 08:40:31 +01:00
Fabian Reinartz 5fb01d41aa Use new Prometheus text format parser 2017-01-16 21:29:53 +01:00
Fabian Reinartz dd0b69fe1b Export ErrNotFound 2017-01-16 14:18:32 +01:00
Fabian Reinartz 1c80c33e72 Fix bug of unsorted postings lists being created 
The former approach created unordered postings list by either
map iteration of new series being unsorted (fixable) or concurrent
writers creating new series interleaved.

We switch back to generating ephemeral references for a single batch.
Newly created series have to be re-set upon the next insert.
 2017-01-13 16:22:20 +01:00
Fabian Reinartz c7f5590a71 Ensure order of postings when adding new series 2017-01-13 15:25:11 +01:00
Fabian Reinartz d970f0256a Add Rollback() and docs to Appender interface 2017-01-12 20:17:49 +01:00
Fabian Reinartz 22db9c3413 Remove old appendBatch methods 2017-01-12 20:04:49 +01:00
Fabian Reinartz fde69dab49 Use buffer pool for head appenders 2017-01-12 20:03:44 +01:00
Fabian Reinartz a317f252b9 Expose series references to clients 
This exposes a reference number of a series represented by a label set
to clients.
Subsequent samples can be directly added via the reference rather than
repeatedly passing in the full labels. This drasitcally speeds up the
append process.

The appender chain uses different sections of the reference number for
assignment to child appenders and invalidating reference numbers as
necessary.

Clients can either pass out reference numbers themselves or have their
own optimized lookup, i.e. by directly associating unparsed metric
descriptors strings with reference numbers.
 2017-01-12 20:00:54 +01:00
Fabian Reinartz 5e028710d5 Add fast past to validation after lock switch 2017-01-12 15:51:08 +01:00
Fabian Reinartz 1b39887baa Revalidate series existance after lock switch 2017-01-11 14:05:58 +01:00
Fabian Reinartz ca5791efbc Simplify creation of new series 2017-01-11 13:58:26 +01:00
Fabian Reinartz 0ca755b4ae Replace single head chunk per series with memSeries 
This adds a memory series holding several chunk to replace
the single head chunk per series so far.
This is necessary for uniform maximum chunk sizes in cases
where some series have higher frequency samples than others.
 2017-01-11 13:02:38 +01:00
Fabian Reinartz c32a94d409 Unexport HeadBlock, export Block interface 2017-01-10 15:41:57 +01:00
Fabian Reinartz d86e8a63c7 Report correct number of appended samples 2017-01-10 11:17:37 +01:00
Fabian Reinartz 29883a18fc Add own Appender() method for DB 2017-01-09 22:54:08 +01:00
Fabian Reinartz 4c4e0c614e Simplify position mapper updating 2017-01-09 19:24:05 +01:00
Fabian Reinartz 0dffd52238 Use page writer in compaction 2017-01-09 18:47:43 +01:00
Fabian Reinartz 89d8467f5c Add missing lock 2017-01-09 18:07:45 +01:00
Fabian Reinartz 8c31c6e934 Make concurrent head chunk reads safe, fix misc races 
This adds a 4 sample buffer to every head chunk. The XOR
compression scheme may edit bytes in place. The minimum size
of a sample is 2 bits. So keeping the last 4 samples in an in-memory
buffer makes it safe to query the preceeding ones while samples
are added
 2017-01-09 16:51:39 +01:00
Fabian Reinartz 1943f8d1bb Fix head block stats races 2017-01-07 18:02:17 +01:00
Fabian Reinartz 6aa922c5a6 Fix races 2017-01-07 16:20:32 +01:00
Fabian Reinartz 300f4e2abf Use separate lock for series creation 
This uses the head block's own lock to only lock if new series were
encountered.
In the general append case we just need to hold a
 2017-01-06 18:10:50 +01:00
Fabian Reinartz 63e12807da Don't update head postings mapper on every append 2017-01-06 16:43:18 +01:00
Fabian Reinartz 71efd2e08d Periodically fsync WAL, make head cut async 2017-01-06 15:18:06 +01:00
Fabian Reinartz 96c2bd249f Handle compaction trigger and reinitializing in DB 2017-01-06 13:03:23 +01:00
Fabian Reinartz 937cdb579c Switch to sequential block names 
This changes block directory names from the int64 timestamp
to sequential numbering.
 2017-01-06 10:45:03 +01:00
Fabian Reinartz 9790aa98ac Add postings wrapper that emits head postings in label set order 
This adds a position mapper that takes series from a head block
in the order they were appended and creates a mapping representing
them in order of their label sets.

Write-repair of the postings list would cause very expensive writing.
Hence, we keep them as they are and only apply the postition mapping
at the very end, after a postings list has been sufficienctly reduced
through intersections etc.
 2017-01-05 16:05:42 +01:00
Fabian Reinartz 5aa7f7cce8 Compact head block into persisted block 2017-01-04 21:11:15 +01:00
Fabian Reinartz 3f72d5d027 Fix last timestamp initialization 
This initializes the chunkDesc's last timestamp to the minimum
value so initial samples with a timestamp of 0 (e.g. in tests)
are not accidentally dropped.
 2017-01-04 14:06:40 +01:00
Fabian Reinartz ac49f8c15e Consolidate persistence and compaction 2017-01-03 16:49:37 +01:00
Fabian Reinartz e7f04d14d5 Lock mmapped files 2017-01-03 10:09:20 +01:00
Fabian Reinartz 91b65b55e7 Run persistence in separate goroutine 2017-01-02 22:24:35 +01:00
Fabian Reinartz a648ef5252 Convert persister into function 2017-01-02 16:58:47 +01:00
Fabian Reinartz dbd2b21d2e Make persistence atomic 2017-01-02 14:41:13 +01:00
Fabian Reinartz beb842a856 Change block interface 
This changes the block interface to directly expose index
and series readers rather than the direct querier
 2017-01-02 11:12:28 +01:00
Fabian Reinartz 5d7ec06e04 Use ChunkMeta in SeriesWriter 2017-01-01 20:01:17 +01:00
Fabian Reinartz c00d17e691 Modify IndexReader API to accomodate compaction 
This changes the IndexReader API to expose plain labels
and chunk meta information instead of a Series interface.
Dropping of irrelevant chunks is moved into the querier.

A LabelIndices method is added to query for existing label
value indices.
 2016-12-31 16:24:04 +01:00
Fabian Reinartz 1e1a37b15b Remove double-reference in chunk hashmap 2016-12-31 10:19:02 +01:00
Fabian Reinartz 675f0886f0 Append to chunks cannot error 2016-12-31 10:10:27 +01:00
Fabian Reinartz 7280533c42 Add basic shard metrics 2016-12-31 09:48:49 +01:00
Fabian Reinartz 201d7687b6 Fix uniqueness of new series 2016-12-26 16:55:32 +01:00
Fabian Reinartz 787199a88e Fix erroneous value assignments 2016-12-22 20:57:00 +01:00
Fabian Reinartz 8aba95048a Rename to OpenHeadBlock 2016-12-22 20:00:24 +01:00
Fabian Reinartz 1dde3b6d31 Add WAL decoder+loading and benchmarks 2016-12-22 15:18:33 +01:00