mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-26 22:19:40 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
476 commits 243 branches 444 tags 649 MiB
Commit graph

48 commits

Author SHA1 Message Date
Fabian Reinartz 8214dc82a7 Remove infinite block in benchmark 2017-09-18 11:20:25 +02:00
Fabian Reinartz 6892fc6dcb Finish old WAL segment async, default to no fsync 
We were still fsyncing while holding the write lock when we cut a new
segment. Given we cannot do anything but logging errors, we might just
as well complete segments asynchronously.

There's not realistic use case where one would fsync after every WAL
entry, thus make the default of a flush interval of 0 to never fsync
which is a much more likely use case.
 2017-09-08 18:41:12 +02:00
Fabian Reinartz c36d574290 Replace single head lock with granular locks 
This adds various new locks to replace the single big lock on
the head. All parts now must be COW as they may be held by clients
after initial retrieval.
Series by ID and hashes are now held in a stripe lock to reduce
contention and total holding time during GC. This should reduce
starvation of readers.
 2017-09-05 14:41:39 +02:00
Fabian Reinartz 3951d8cc29 Fix selection of compactable blocks 2017-08-04 12:02:34 +02:00
Fabian Reinartz 9c4235532e Fix compaction selection after creating new heads 
This fixes the case where between block creations no compaction
plans are ran. We were not compacting anything in these
cases since the on creation the most recent head block always had
a high timestamp of 0.
 2017-07-13 16:15:13 +02:00
Goutham Veeramachaneni 9a2ab732b5 Choose ranges by retention and min-block-duration 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 17:30:34 +02:00
Fabian Reinartz 05e411a8eb Improve heuristic to spread chunks across block 2017-06-08 11:30:32 +02:00
Fabian Reinartz 39df7e2bba Switch blocks to ULID directories, drop sequenc numbers 2017-05-18 16:09:30 +02:00
Fabian Reinartz 285bc07030 Switch append refs to string 2017-05-18 10:56:57 +02:00
Fabian Reinartz c8438cfc81 Add mutex profiling to benchmark 2017-05-14 11:51:56 +02:00
Fabian Reinartz 778103b450 Add liecence file and headers 2017-04-10 20:59:45 +02:00
Fabian Reinartz 10c7c9acbe Adjust import names to new repository organisation 2017-04-04 11:27:26 +02:00
Goutham Veeramachaneni 761e4768f3
Lint and Vet Fixes 2017-03-19 21:35:01 +05:30
Fabian Reinartz 87805fb83f Remove Partitioned* code 2017-03-06 17:34:49 +01:00
Fabian Reinartz cc0a7c8279 Create alloc and inuse space heap profile 2017-03-02 21:54:30 +01:00
Fabian Reinartz ffb24a98f4 Add missing unlock, run debug endpoint in benchmark 2017-03-02 21:54:02 +01:00
Fabian Reinartz db5c88ea9a Misc compaction fixes 2017-02-28 08:40:51 +01:00
Fabian Reinartz b281e4e39b Accept prometheus.Registerer in constructor 2017-02-28 07:26:24 +01:00
Fabian Reinartz a3b47c4929 Create default logger for DB 2017-02-19 16:04:37 +01:00
Fabian Reinartz 9c7a88223e Add full encode/decode WAL cycle test 2017-02-14 21:55:50 -08:00
Fabian Reinartz 79944a5912 Break out WAL into segment files 2017-02-13 23:53:19 -08:00
Fabian Reinartz 9c76624df2 Add initial retention cutoff 2017-02-09 17:54:26 -08:00
Fabian Reinartz 012cf4ef25 Count writer references on head blocks 2017-02-04 11:53:52 +01:00
Fabian Reinartz 5a1c8eaa0e Fix missing appends after reference lookups 2017-02-02 11:10:17 +01:00
Fabian Reinartz 30efe4a58c Support writing to multiple head blocks 
This is an initial (and hacky) first pass on allowing
appending to multiple blocks simultaniously to avoid
dropping samples right after cutting a new head block.
It's also required for cases like the PGW, where a scrape may
contain varying timestamps.
 2017-02-01 15:57:28 +01:00
Fabian Reinartz c20cc44b06 Add docs, write sequence number to meta.json 2017-01-29 08:11:47 +01:00
Fabian Reinartz 5fb01d41aa Use new Prometheus text format parser 2017-01-16 21:29:53 +01:00
Fabian Reinartz fde69dab49 Use buffer pool for head appenders 2017-01-12 20:03:44 +01:00
Fabian Reinartz a317f252b9 Expose series references to clients 
This exposes a reference number of a series represented by a label set
to clients.
Subsequent samples can be directly added via the reference rather than
repeatedly passing in the full labels. This drasitcally speeds up the
append process.

The appender chain uses different sections of the reference number for
assignment to child appenders and invalidating reference numbers as
necessary.

Clients can either pass out reference numbers themselves or have their
own optimized lookup, i.e. by directly associating unparsed metric
descriptors strings with reference numbers.
 2017-01-12 20:00:54 +01:00
Fabian Reinartz 80affd98a8 Add barrier to benchmark writer 
This adds a barrier to avoid issues with unfair goroutine scheduling
that causes some fake scrapers to run away from the other ones.
 2017-01-11 13:01:30 +01:00
Fabian Reinartz c32a94d409 Unexport HeadBlock, export Block interface 2017-01-10 15:41:57 +01:00
Fabian Reinartz d86e8a63c7 Report correct number of appended samples 2017-01-10 11:17:37 +01:00
Fabian Reinartz 300f4e2abf Use separate lock for series creation 
This uses the head block's own lock to only lock if new series were
encountered.
In the general append case we just need to hold a
 2017-01-06 18:10:50 +01:00
Fabian Reinartz c61b310210 Naive size-based compaction 
This adds naive compaction that tries to compact three
blocks of roughly equal size.
It decides based on samples present in a block and has no
safety measures considering the actual file size.
 2017-01-06 13:53:05 +01:00
Fabian Reinartz 3ed2c2a14b Rename Partition to regular DB, DB to PartitionedDB 2017-01-06 11:40:09 +01:00
Fabian Reinartz cabd7e4ebc Add small testdata file 2016-12-22 19:15:37 +01:00
Fabian Reinartz ede733ab6c Extract labels package 2016-12-21 09:39:01 +01:00
Fabian Reinartz ee217adc7e Redfine append interface, remove old Prometheus storage from bench 2016-12-21 00:02:37 +01:00
Fabian Reinartz e561c91d53 Implement proper buffered iterator 
This adds a proper duration based lookback buffer for series iterators
to allow advancing sequentially while remaining able to calculate time
aggregating functions such as `rate` backwards.

It uses an array ring buffer to minimize heap allocations for
potentially hundreds of thousands of series for a single query.
 2016-12-14 21:14:44 +01:00
Fabian Reinartz 3a528c3078 Write plain postings list index 2016-12-10 09:44:00 +01:00
Fabian Reinartz 40a451694f Refactor persistence into interfaces 2016-12-09 20:45:46 +01:00
Fabian Reinartz 74f8dfd95d Persist blocks periodically 2016-12-09 13:41:38 +01:00
Fabian Reinartz 8aa99a3ebd misc 2016-12-09 10:00:14 +01:00
Fabian Reinartz 63b887eb62 Add Makefile 2016-12-08 12:00:05 +01:00
Fabian Reinartz b845f8d3a1 Reduce test data allocations 2016-12-08 11:59:54 +01:00
Fabian Reinartz ce82bdb71a Add write benchmark utility 2016-12-07 17:30:10 +01:00
Fabian Reinartz ff29705571 Delete old benchmarking tool 2016-12-04 13:15:49 +01:00
Fabian Reinartz 40c27a1c2f Add benchmark utility 2016-11-15 10:35:42 +01:00