Fabian Reinartz
1a35e54450
Add chained iterator, skipping seek added
2016-12-15 15:23:15 +01:00
Fabian Reinartz
205edd2da9
Add byte/string equality check benchmark
2016-12-15 12:15:54 +01:00
Fabian Reinartz
b2f1db5666
Add unsafe string and slice conversions
2016-12-15 11:56:41 +01:00
Fabian Reinartz
9ceed5378e
Fix missing bound checks, off-by-ones, typos
2016-12-15 11:24:20 +01:00
Fabian Reinartz
5424a0cf75
Rename SeriesShard to Shard
2016-12-15 08:36:09 +01:00
Fabian Reinartz
9873e18b75
Add loading of persisted blocks
2016-12-15 08:36:07 +01:00
Fabian Reinartz
d56b281006
Rename Iterator to Postings
2016-12-15 07:05:29 +01:00
Fabian Reinartz
c1acd3fe85
Advance buffered iterator correctly on seek
2016-12-14 21:43:27 +01:00
Fabian Reinartz
e561c91d53
Implement proper buffered iterator
...
This adds a proper duration based lookback buffer for series iterators
to allow advancing sequentially while remaining able to calculate time
aggregating functions such as `rate` backwards.
It uses an array ring buffer to minimize heap allocations for
potentially hundreds of thousands of series for a single query.
2016-12-14 21:14:44 +01:00
Fabian Reinartz
ca89080128
Misc fixes for initial Prometheus integration
2016-12-14 18:38:46 +01:00
Fabian Reinartz
725385ea05
Fix compareLabels, add test
2016-12-14 15:47:05 +01:00
Fabian Reinartz
fc992fafc2
Change querier interface, initial implementations
2016-12-14 15:39:23 +01:00
Fabian Reinartz
6eeb0ef01c
Add queriers and partial mocks
2016-12-13 15:26:58 +01:00
Fabian Reinartz
9b400b4c58
Add chunk based series iterator
2016-12-12 19:12:55 +01:00
Fabian Reinartz
b334c3ade8
Write chunk skiplist and add series reader
2016-12-12 15:39:55 +01:00
Fabian Reinartz
ae379f385b
Fix label index write and add read path
2016-12-12 11:38:43 +01:00
Fabian Reinartz
10943b6d88
Add initial index reader implementation
2016-12-12 08:12:19 +01:00
Fabian Reinartz
70a0224f19
Change chunk sample number to BigEndian
2016-12-12 08:11:53 +01:00
Fabian Reinartz
81b4d570ad
Add series file reader
2016-12-11 15:54:25 +01:00
Fabian Reinartz
5e02e28f9c
Add proper mmap calls
2016-12-11 15:49:36 +01:00
Fabian Reinartz
8425df035d
Fix hashmap serialization
2016-12-11 15:49:24 +01:00
Fabian Reinartz
14dbc59f2b
cleanup and switching removal of unsafe calls.
2016-12-10 18:09:57 +01:00
Fabian Reinartz
eb9af096f9
Write hashmap pointers, simplify section writer
2016-12-10 10:13:54 +01:00
Fabian Reinartz
3a528c3078
Write plain postings list index
2016-12-10 09:44:00 +01:00
Fabian Reinartz
4eba874b04
Factor out section writer
2016-12-09 22:36:31 +01:00
Fabian Reinartz
0b77a3dafc
Write series references into index
2016-12-09 22:27:43 +01:00
Fabian Reinartz
55b36ab413
Index persistence fixes, write label index hash table
2016-12-09 22:12:16 +01:00
Fabian Reinartz
8cbc95c316
Write label value indices
2016-12-09 21:40:38 +01:00
Fabian Reinartz
1e0edf367b
Write index with symbol table
2016-12-09 21:23:34 +01:00
Fabian Reinartz
40a451694f
Refactor persistence into interfaces
2016-12-09 20:45:46 +01:00
Fabian Reinartz
62f9dc311c
misc
2016-12-09 16:54:38 +01:00
Fabian Reinartz
74f8dfd95d
Persist blocks periodically
2016-12-09 13:41:38 +01:00
Fabian Reinartz
0cf8bb9e53
Move sub-indexes into single index structure
2016-12-09 10:41:51 +01:00
Fabian Reinartz
8aa99a3ebd
misc
2016-12-09 10:00:14 +01:00
Fabian Reinartz
2c34a15fe6
Add initial seriailization of block data
2016-12-08 17:43:10 +01:00
Fabian Reinartz
3ef7da33c8
Restructure files
2016-12-08 12:21:03 +01:00
Fabian Reinartz
63b887eb62
Add Makefile
2016-12-08 12:00:05 +01:00
Fabian Reinartz
b845f8d3a1
Reduce test data allocations
2016-12-08 11:59:54 +01:00
Fabian Reinartz
ce82bdb71a
Add write benchmark utility
2016-12-07 17:30:10 +01:00
Fabian Reinartz
52276c6966
Bucket samples before appending.
...
This pre-sorts samples into buckets before appending them to reduce
locking of shards.
2016-12-07 17:10:49 +01:00
Fabian Reinartz
c5945177fb
chunks: helper for bit range
2016-12-07 15:37:37 +01:00
Fabian Reinartz
9b459458d0
Docs and interface definitions
2016-12-05 21:26:19 +01:00
Fabian Reinartz
83574b1565
Add new interfaces and skeleton
2016-12-04 13:16:11 +01:00
Fabian Reinartz
ff29705571
Delete old benchmarking tool
2016-12-04 13:15:49 +01:00
Fabian Reinartz
6f93a699e6
wipe
2016-12-02 17:49:05 +01:00
Fabian Reinartz
0b6d621471
chunks: bring back lead/trail reuse, truncate incomplete sample
2016-11-30 23:01:01 +01:00
Fabian Reinartz
8c48dc2ca5
chunks: varint encoding in first full 64bit numbers
...
This saves about 7 bytes per chunk
2016-11-30 22:14:23 +01:00
Fabian Reinartz
5e76fd3126
chunks: drop shift, remove branch
...
This removes the need for the shift value and just calculates
everything on the fly through count.
Removes a branch condition.
2016-11-30 21:50:39 +01:00
Fabian Reinartz
c9ee572709
chunks: don't mutate in bstream iterator
...
This replaces mutation of underlying bytes in the iterated slice
with a shift counter, which is used when reading the head byte.
This is avoids having to copy the entire slice for every new iterator.
2016-11-30 19:39:22 +01:00
Fabian Reinartz
f392c01c12
chunks: add randomized test
2016-11-29 22:43:24 +01:00