77 commits

Author	SHA1	Message	Date
Fabian Reinartz	9c76624df2	Add initial retention cutoff	2017-02-09 17:54:26 -08:00
Fabian Reinartz	012cf4ef25	Count writer references on head blocks	2017-02-04 11:53:52 +01:00
Fabian Reinartz	5a1c8eaa0e	Fix missing appends after reference lookups	2017-02-02 11:10:17 +01:00
Fabian Reinartz	30efe4a58c	Support writing to multiple head blocks ... This is an initial (and hacky) first pass on allowing appending to multiple blocks simultaniously to avoid dropping samples right after cutting a new head block. It's also required for cases like the PGW, where a scrape may contain varying timestamps.	2017-02-01 15:57:28 +01:00
Fabian Reinartz	c20cc44b06	Add docs, write sequence number to meta.json	2017-01-29 08:11:47 +01:00
Fabian Reinartz	5fb01d41aa	Use new Prometheus text format parser	2017-01-16 21:29:53 +01:00
Fabian Reinartz	fde69dab49	Use buffer pool for head appenders	2017-01-12 20:03:44 +01:00
Fabian Reinartz	a317f252b9	Expose series references to clients ... This exposes a reference number of a series represented by a label set to clients. Subsequent samples can be directly added via the reference rather than repeatedly passing in the full labels. This drasitcally speeds up the append process. The appender chain uses different sections of the reference number for assignment to child appenders and invalidating reference numbers as necessary. Clients can either pass out reference numbers themselves or have their own optimized lookup, i.e. by directly associating unparsed metric descriptors strings with reference numbers.	2017-01-12 20:00:54 +01:00
Fabian Reinartz	80affd98a8	Add barrier to benchmark writer ... This adds a barrier to avoid issues with unfair goroutine scheduling that causes some fake scrapers to run away from the other ones.	2017-01-11 13:01:30 +01:00
Fabian Reinartz	c32a94d409	Unexport HeadBlock, export Block interface	2017-01-10 15:41:57 +01:00
Fabian Reinartz	d86e8a63c7	Report correct number of appended samples	2017-01-10 11:17:37 +01:00
Fabian Reinartz	300f4e2abf	Use separate lock for series creation ... This uses the head block's own lock to only lock if new series were encountered. In the general append case we just need to hold a	2017-01-06 18:10:50 +01:00
Fabian Reinartz	c61b310210	Naive size-based compaction ... This adds naive compaction that tries to compact three blocks of roughly equal size. It decides based on samples present in a block and has no safety measures considering the actual file size.	2017-01-06 13:53:05 +01:00
Fabian Reinartz	3ed2c2a14b	Rename Partition to regular DB, DB to PartitionedDB	2017-01-06 11:40:09 +01:00
Fabian Reinartz	cabd7e4ebc	Add small testdata file	2016-12-22 19:15:37 +01:00
Fabian Reinartz	ede733ab6c	Extract labels package	2016-12-21 09:39:01 +01:00
Fabian Reinartz	ee217adc7e	Redfine append interface, remove old Prometheus storage from bench	2016-12-21 00:02:37 +01:00
Fabian Reinartz	e561c91d53	Implement proper buffered iterator ... This adds a proper duration based lookback buffer for series iterators to allow advancing sequentially while remaining able to calculate time aggregating functions such as `rate` backwards. It uses an array ring buffer to minimize heap allocations for potentially hundreds of thousands of series for a single query.	2016-12-14 21:14:44 +01:00
Fabian Reinartz	3a528c3078	Write plain postings list index	2016-12-10 09:44:00 +01:00
Fabian Reinartz	40a451694f	Refactor persistence into interfaces	2016-12-09 20:45:46 +01:00
Fabian Reinartz	74f8dfd95d	Persist blocks periodically	2016-12-09 13:41:38 +01:00
Fabian Reinartz	8aa99a3ebd	misc	2016-12-09 10:00:14 +01:00
Fabian Reinartz	63b887eb62	Add Makefile	2016-12-08 12:00:05 +01:00
Fabian Reinartz	b845f8d3a1	Reduce test data allocations	2016-12-08 11:59:54 +01:00
Fabian Reinartz	ce82bdb71a	Add write benchmark utility	2016-12-07 17:30:10 +01:00
Fabian Reinartz	ff29705571	Delete old benchmarking tool	2016-12-04 13:15:49 +01:00
Fabian Reinartz	40c27a1c2f	Add benchmark utility	2016-11-15 10:35:42 +01:00