Commit graph

18 commits

Author SHA1 Message Date
Fabian Reinartz fde69dab49 Use buffer pool for head appenders 2017-01-12 20:03:44 +01:00
Fabian Reinartz a317f252b9 Expose series references to clients
This exposes a reference number of a series represented by a label set
to clients.
Subsequent samples can be directly added via the reference rather than
repeatedly passing in the full labels. This drasitcally speeds up the
append process.

The appender chain uses different sections of the reference number for
assignment to child appenders and invalidating reference numbers as
necessary.

Clients can either pass out reference numbers themselves or have their
own optimized lookup, i.e. by directly associating unparsed metric
descriptors strings with reference numbers.
2017-01-12 20:00:54 +01:00
Fabian Reinartz 80affd98a8 Add barrier to benchmark writer
This adds a barrier to avoid issues with unfair goroutine scheduling
that causes some fake scrapers to run away from the other ones.
2017-01-11 13:01:30 +01:00
Fabian Reinartz d86e8a63c7 Report correct number of appended samples 2017-01-10 11:17:37 +01:00
Fabian Reinartz 300f4e2abf Use separate lock for series creation
This uses the head block's own lock to only lock if new series were
encountered.
In the general append case we just need to hold a
2017-01-06 18:10:50 +01:00
Fabian Reinartz c61b310210 Naive size-based compaction
This adds naive compaction that tries to compact three
blocks of roughly equal size.
It decides based on samples present in a block and has no
safety measures considering the actual file size.
2017-01-06 13:53:05 +01:00
Fabian Reinartz 3ed2c2a14b Rename Partition to regular DB, DB to PartitionedDB 2017-01-06 11:40:09 +01:00
Fabian Reinartz ede733ab6c Extract labels package 2016-12-21 09:39:01 +01:00
Fabian Reinartz ee217adc7e Redfine append interface, remove old Prometheus storage from bench 2016-12-21 00:02:37 +01:00
Fabian Reinartz e561c91d53 Implement proper buffered iterator
This adds a proper duration based lookback buffer for series iterators
to allow advancing sequentially while remaining able to calculate time
aggregating functions such as `rate` backwards.

It uses an array ring buffer to minimize heap allocations for
potentially hundreds of thousands of series for a single query.
2016-12-14 21:14:44 +01:00
Fabian Reinartz 3a528c3078 Write plain postings list index 2016-12-10 09:44:00 +01:00
Fabian Reinartz 40a451694f Refactor persistence into interfaces 2016-12-09 20:45:46 +01:00
Fabian Reinartz 74f8dfd95d Persist blocks periodically 2016-12-09 13:41:38 +01:00
Fabian Reinartz 8aa99a3ebd misc 2016-12-09 10:00:14 +01:00
Fabian Reinartz b845f8d3a1 Reduce test data allocations 2016-12-08 11:59:54 +01:00
Fabian Reinartz ce82bdb71a Add write benchmark utility 2016-12-07 17:30:10 +01:00
Fabian Reinartz ff29705571 Delete old benchmarking tool 2016-12-04 13:15:49 +01:00
Fabian Reinartz 40c27a1c2f Add benchmark utility 2016-11-15 10:35:42 +01:00