mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-26 22:19:40 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
445 commits 243 branches 444 tags 649 MiB
Commit graph

445 commits

This branch
This branch
All branches
Author SHA1 Message Date
Fabian Reinartz 3e569bc964 Optimize memory allocations in writer 2017-03-02 23:35:02 +01:00
Fabian Reinartz cc0a7c8279 Create alloc and inuse space heap profile 2017-03-02 21:54:30 +01:00
Fabian Reinartz ffb24a98f4 Add missing unlock, run debug endpoint in benchmark 2017-03-02 21:54:02 +01:00
Fabian Reinartz 21b97d1e04 Fix compaction selection procedure 2017-03-02 15:10:13 +01:00
Fabian Reinartz 327e07e8d0 Properly sync moves and removes in compactor 2017-03-02 14:35:06 +01:00
Fabian Reinartz 92120448c2 Properly cleanup compacted dirs, fixes, docs 2017-03-02 14:32:09 +01:00
Fabian Reinartz 2c3e778d90 Compactor interface 2017-03-02 09:17:30 +01:00
Fabian Reinartz 2c3b56350a Write meta.json file atomically 2017-03-01 17:19:57 +01:00
Fabian Reinartz 6c4217276f Periodically trigger compaction 
This addresses an issue where the compaction triggered on cutting
a new block doesn't find anything as the writers are still active on the
block that should be ready for compaction.
 2017-03-01 16:52:21 +01:00
Fabian Reinartz db5c88ea9a Misc compaction fixes 2017-02-28 08:40:51 +01:00
Fabian Reinartz b281e4e39b Accept prometheus.Registerer in constructor 2017-02-28 07:26:24 +01:00
Fabian Reinartz 306831f151 Add per-block state ULID 2017-02-27 10:46:15 +01:00
Fabian Reinartz 9a5dfadb09 Tweak IndexWriter interface, serialize directly into dir 2017-02-25 07:24:20 +01:00
Fabian Reinartz 78780cd2ba Segment chunk file 
This adds write path support for segmented chunk data files.
Files of 512MB are pre-allocated and written to. If the file size
is exceeded, the next file is started. On completion, files
are truncated to their final size.
 2017-02-23 10:50:22 +01:00
Fabian Reinartz a3b47c4929 Create default logger for DB 2017-02-19 16:04:37 +01:00
Fabian Reinartz f734773214 Replace per-file locking with single PID lock file 
File locks have a multitude of problems that make them hard to use
correctly. As they are just advisory, they are only meaningful to
prevent accidents like running the same process twice.
A simple PID file lock works reliably in those cases and is simpler.
 2017-02-19 13:01:19 +01:00
Fabian Reinartz c808928b90 SeriesWriter -> ChunkWriter 
This commit simplifies a SeriesWriter into a ChunkWriter and detaches
it entirely from the notion of a series and the series index.
 2017-02-19 11:27:31 +01:00
Fabian Reinartz a3d042b54e Support multiple chunk files in read path 2017-02-18 17:33:20 +01:00
Fabian Reinartz afa084920c Use Castagnoli polynomial for CRC32s; cache them 2017-02-15 21:13:17 -08:00
Fabian Reinartz 9c7a88223e Add full encode/decode WAL cycle test 2017-02-14 21:55:50 -08:00
Fabian Reinartz 2c97428a79 Add basic WAL tests 2017-02-14 16:12:56 -08:00
Fabian Reinartz 84e8027a8e Close consumed WAL segments, verify checksums on read 2017-02-14 00:42:06 -08:00
Fabian Reinartz eeb03e97e2 Add headers and format flag for WAL segments 2017-02-14 00:24:53 -08:00
Fabian Reinartz 79944a5912 Break out WAL into segment files 2017-02-13 23:53:19 -08:00
Fabian Reinartz 987a90d149 Don't delete files if retention duration is 0 2017-02-11 11:49:17 -08:00
Fabian Reinartz 9c76624df2 Add initial retention cutoff 2017-02-09 17:54:26 -08:00
Fabian Reinartz f1435f2e2c Track appended samples properly in metric 2017-02-08 16:13:16 -08:00
Fabian Reinartz 012cf4ef25 Count writer references on head blocks 2017-02-04 11:53:52 +01:00
Fabian Reinartz 5a1c8eaa0e Fix missing appends after reference lookups 2017-02-02 11:10:17 +01:00
Fabian Reinartz f556036037 Close blocks before overwriting with compacted one 
This prevents the meta.json file of a compacted block
to be overwritten with data of the old block.
 2017-02-02 09:32:06 +01:00
Fabian Reinartz ac5229e1b4 Correctly write empty blocks and extend appenders by new blocks 2017-02-02 07:58:54 +01:00
Fabian Reinartz 42fa342229 Improve multi-head handling 
This improves handling of multiple head blocks. Configuration is
simplified to specify the number of concurrently appendable blocks.
 2017-02-01 21:35:21 +01:00
Fabian Reinartz 30efe4a58c Support writing to multiple head blocks 
This is an initial (and hacky) first pass on allowing
appending to multiple blocks simultaniously to avoid
dropping samples right after cutting a new head block.
It's also required for cases like the PGW, where a scrape may
contain varying timestamps.
 2017-02-01 15:57:28 +01:00
Fabian Reinartz a61a31a5d7 compaction: add fast-path for compacting mem blocks 2017-01-30 09:49:36 +01:00
Fabian Reinartz c20cc44b06 Add docs, write sequence number to meta.json 2017-01-29 08:11:47 +01:00
Fabian Reinartz 30d8866c13 Limit compaction range, make cut and compact concurrent 2017-01-20 07:58:19 +01:00
Fabian Reinartz 67d185ceb9 Compact based on compaction generation 2017-01-19 19:45:52 +01:00
Fabian Reinartz 472c618c39 Drop out-of-bound samples 2017-01-19 15:03:57 +01:00
Fabian Reinartz d4779b374c Properly track and write meta file 2017-01-19 14:01:38 +01:00
Fabian Reinartz 9ddbd64d00 Move stats into meta.json file, cleanup, docs 2017-01-19 11:22:47 +01:00
Fabian Reinartz 2f02f86b62 Fix WAL tests 2017-01-19 08:48:11 +01:00
Fabian Reinartz e006bc6dc6 Improve error messages, create regular dir for block 2017-01-19 08:40:15 +01:00
Fabian Reinartz d2322f6095 Improve compaction processing 2017-01-18 06:18:32 +01:00
Fabian Reinartz 5ceca3c810 Write to WAL before appending to memory storage 2017-01-17 16:33:58 +01:00
Fabian Reinartz 343dd9d94c Fix wrong byte size in WAL base ref 2017-01-17 08:40:31 +01:00
Fabian Reinartz 5fb01d41aa Use new Prometheus text format parser 2017-01-16 21:29:53 +01:00
Fabian Reinartz dd0b69fe1b Export ErrNotFound 2017-01-16 14:18:32 +01:00
Fabian Reinartz 9cf49f68e9 wal: use larger buffer 2017-01-16 14:18:25 +01:00
Fabian Reinartz 1c80c33e72 Fix bug of unsorted postings lists being created 
The former approach created unordered postings list by either
map iteration of new series being unsorted (fixable) or concurrent
writers creating new series interleaved.

We switch back to generating ephemeral references for a single batch.
Newly created series have to be re-set upon the next insert.
 2017-01-13 16:22:20 +01:00
Fabian Reinartz c7f5590a71 Ensure order of postings when adding new series 2017-01-13 15:25:11 +01:00