Fabian Reinartz
05e411a8eb
Improve heuristic to spread chunks across block
2017-06-08 11:30:32 +02:00
Fabian Reinartz
39df7e2bba
Switch blocks to ULID directories, drop sequenc numbers
2017-05-18 16:09:30 +02:00
Fabian Reinartz
285bc07030
Switch append refs to string
2017-05-18 10:56:57 +02:00
Fabian Reinartz
c8438cfc81
Add mutex profiling to benchmark
2017-05-14 11:51:56 +02:00
Fabian Reinartz
778103b450
Add liecence file and headers
2017-04-10 20:59:45 +02:00
Fabian Reinartz
10c7c9acbe
Adjust import names to new repository organisation
2017-04-04 11:27:26 +02:00
Goutham Veeramachaneni
761e4768f3
Lint and Vet Fixes
2017-03-19 21:35:01 +05:30
Fabian Reinartz
87805fb83f
Remove Partitioned* code
2017-03-06 17:34:49 +01:00
Fabian Reinartz
cc0a7c8279
Create alloc and inuse space heap profile
2017-03-02 21:54:30 +01:00
Fabian Reinartz
ffb24a98f4
Add missing unlock, run debug endpoint in benchmark
2017-03-02 21:54:02 +01:00
Fabian Reinartz
db5c88ea9a
Misc compaction fixes
2017-02-28 08:40:51 +01:00
Fabian Reinartz
b281e4e39b
Accept prometheus.Registerer in constructor
2017-02-28 07:26:24 +01:00
Fabian Reinartz
a3b47c4929
Create default logger for DB
2017-02-19 16:04:37 +01:00
Fabian Reinartz
9c7a88223e
Add full encode/decode WAL cycle test
2017-02-14 21:55:50 -08:00
Fabian Reinartz
79944a5912
Break out WAL into segment files
2017-02-13 23:53:19 -08:00
Fabian Reinartz
9c76624df2
Add initial retention cutoff
2017-02-09 17:54:26 -08:00
Fabian Reinartz
012cf4ef25
Count writer references on head blocks
2017-02-04 11:53:52 +01:00
Fabian Reinartz
5a1c8eaa0e
Fix missing appends after reference lookups
2017-02-02 11:10:17 +01:00
Fabian Reinartz
30efe4a58c
Support writing to multiple head blocks
...
This is an initial (and hacky) first pass on allowing
appending to multiple blocks simultaniously to avoid
dropping samples right after cutting a new head block.
It's also required for cases like the PGW, where a scrape may
contain varying timestamps.
2017-02-01 15:57:28 +01:00
Fabian Reinartz
c20cc44b06
Add docs, write sequence number to meta.json
2017-01-29 08:11:47 +01:00
Fabian Reinartz
5fb01d41aa
Use new Prometheus text format parser
2017-01-16 21:29:53 +01:00
Fabian Reinartz
fde69dab49
Use buffer pool for head appenders
2017-01-12 20:03:44 +01:00
Fabian Reinartz
a317f252b9
Expose series references to clients
...
This exposes a reference number of a series represented by a label set
to clients.
Subsequent samples can be directly added via the reference rather than
repeatedly passing in the full labels. This drasitcally speeds up the
append process.
The appender chain uses different sections of the reference number for
assignment to child appenders and invalidating reference numbers as
necessary.
Clients can either pass out reference numbers themselves or have their
own optimized lookup, i.e. by directly associating unparsed metric
descriptors strings with reference numbers.
2017-01-12 20:00:54 +01:00
Fabian Reinartz
80affd98a8
Add barrier to benchmark writer
...
This adds a barrier to avoid issues with unfair goroutine scheduling
that causes some fake scrapers to run away from the other ones.
2017-01-11 13:01:30 +01:00
Fabian Reinartz
c32a94d409
Unexport HeadBlock, export Block interface
2017-01-10 15:41:57 +01:00
Fabian Reinartz
d86e8a63c7
Report correct number of appended samples
2017-01-10 11:17:37 +01:00
Fabian Reinartz
300f4e2abf
Use separate lock for series creation
...
This uses the head block's own lock to only lock if new series were
encountered.
In the general append case we just need to hold a
2017-01-06 18:10:50 +01:00
Fabian Reinartz
c61b310210
Naive size-based compaction
...
This adds naive compaction that tries to compact three
blocks of roughly equal size.
It decides based on samples present in a block and has no
safety measures considering the actual file size.
2017-01-06 13:53:05 +01:00
Fabian Reinartz
3ed2c2a14b
Rename Partition to regular DB, DB to PartitionedDB
2017-01-06 11:40:09 +01:00
Fabian Reinartz
cabd7e4ebc
Add small testdata file
2016-12-22 19:15:37 +01:00
Fabian Reinartz
ede733ab6c
Extract labels package
2016-12-21 09:39:01 +01:00
Fabian Reinartz
ee217adc7e
Redfine append interface, remove old Prometheus storage from bench
2016-12-21 00:02:37 +01:00
Fabian Reinartz
e561c91d53
Implement proper buffered iterator
...
This adds a proper duration based lookback buffer for series iterators
to allow advancing sequentially while remaining able to calculate time
aggregating functions such as `rate` backwards.
It uses an array ring buffer to minimize heap allocations for
potentially hundreds of thousands of series for a single query.
2016-12-14 21:14:44 +01:00
Fabian Reinartz
3a528c3078
Write plain postings list index
2016-12-10 09:44:00 +01:00
Fabian Reinartz
40a451694f
Refactor persistence into interfaces
2016-12-09 20:45:46 +01:00
Fabian Reinartz
74f8dfd95d
Persist blocks periodically
2016-12-09 13:41:38 +01:00
Fabian Reinartz
8aa99a3ebd
misc
2016-12-09 10:00:14 +01:00
Fabian Reinartz
63b887eb62
Add Makefile
2016-12-08 12:00:05 +01:00
Fabian Reinartz
b845f8d3a1
Reduce test data allocations
2016-12-08 11:59:54 +01:00
Fabian Reinartz
ce82bdb71a
Add write benchmark utility
2016-12-07 17:30:10 +01:00
Fabian Reinartz
ff29705571
Delete old benchmarking tool
2016-12-04 13:15:49 +01:00
Fabian Reinartz
40c27a1c2f
Add benchmark utility
2016-11-15 10:35:42 +01:00