Commit graph

1047 commits

Author SHA1 Message Date
Fabian Reinartz 10943b6d88 Add initial index reader implementation 2016-12-12 08:12:19 +01:00
Fabian Reinartz 70a0224f19 Change chunk sample number to BigEndian 2016-12-12 08:11:53 +01:00
Fabian Reinartz 81b4d570ad Add series file reader 2016-12-11 15:54:25 +01:00
Fabian Reinartz 5e02e28f9c Add proper mmap calls 2016-12-11 15:49:36 +01:00
Fabian Reinartz 8425df035d Fix hashmap serialization 2016-12-11 15:49:24 +01:00
Fabian Reinartz 14dbc59f2b cleanup and switching removal of unsafe calls. 2016-12-10 18:09:57 +01:00
Fabian Reinartz eb9af096f9 Write hashmap pointers, simplify section writer 2016-12-10 10:13:54 +01:00
Fabian Reinartz 3a528c3078 Write plain postings list index 2016-12-10 09:44:00 +01:00
Fabian Reinartz 4eba874b04 Factor out section writer 2016-12-09 22:36:31 +01:00
Fabian Reinartz 0b77a3dafc Write series references into index 2016-12-09 22:27:43 +01:00
Fabian Reinartz 55b36ab413 Index persistence fixes, write label index hash table 2016-12-09 22:12:16 +01:00
Fabian Reinartz 8cbc95c316 Write label value indices 2016-12-09 21:40:38 +01:00
Fabian Reinartz 1e0edf367b Write index with symbol table 2016-12-09 21:23:34 +01:00
Fabian Reinartz 40a451694f Refactor persistence into interfaces 2016-12-09 20:45:46 +01:00
Fabian Reinartz 62f9dc311c misc 2016-12-09 16:54:38 +01:00
Fabian Reinartz 74f8dfd95d Persist blocks periodically 2016-12-09 13:41:38 +01:00
Fabian Reinartz 0cf8bb9e53 Move sub-indexes into single index structure 2016-12-09 10:41:51 +01:00
Fabian Reinartz 8aa99a3ebd misc 2016-12-09 10:00:14 +01:00
Fabian Reinartz 2c34a15fe6 Add initial seriailization of block data 2016-12-08 17:43:10 +01:00
Fabian Reinartz 3ef7da33c8 Restructure files 2016-12-08 12:21:03 +01:00
Fabian Reinartz 63b887eb62 Add Makefile 2016-12-08 12:00:05 +01:00
Fabian Reinartz b845f8d3a1 Reduce test data allocations 2016-12-08 11:59:54 +01:00
Fabian Reinartz ce82bdb71a Add write benchmark utility 2016-12-07 17:30:10 +01:00
Fabian Reinartz 52276c6966 Bucket samples before appending.
This pre-sorts samples into buckets before appending them to reduce
locking of shards.
2016-12-07 17:10:49 +01:00
Fabian Reinartz c5945177fb chunks: helper for bit range 2016-12-07 15:37:37 +01:00
Fabian Reinartz 9b459458d0 Docs and interface definitions 2016-12-05 21:26:19 +01:00
Fabian Reinartz 83574b1565 Add new interfaces and skeleton 2016-12-04 13:16:11 +01:00
Fabian Reinartz ff29705571 Delete old benchmarking tool 2016-12-04 13:15:49 +01:00
Fabian Reinartz 6f93a699e6 wipe 2016-12-02 17:49:05 +01:00
Fabian Reinartz 0b6d621471 chunks: bring back lead/trail reuse, truncate incomplete sample 2016-11-30 23:01:01 +01:00
Fabian Reinartz 8c48dc2ca5 chunks: varint encoding in first full 64bit numbers
This saves about 7 bytes per chunk
2016-11-30 22:14:23 +01:00
Fabian Reinartz 5e76fd3126 chunks: drop shift, remove branch
This removes the need for the shift value and just calculates
everything on the fly through count.
Removes a branch condition.
2016-11-30 21:50:39 +01:00
Fabian Reinartz c9ee572709 chunks: don't mutate in bstream iterator
This replaces mutation of underlying bytes in the iterated slice
with a shift counter, which is used when reading the head byte.
This is avoids having to copy the entire slice for every new iterator.
2016-11-30 19:39:22 +01:00
Fabian Reinartz f392c01c12 chunks: add randomized test 2016-11-29 22:43:24 +01:00
Fabian Reinartz fa181a34c1 chunks: cleanup anything but xor encoding
xor encoding is fast enough for our purposes and provides
very good compression.
We remove all other ones that partially don't support floats
for the sake of simplicity.
2016-11-29 22:02:58 +01:00
Fabian Reinartz e67cf768dc chunks: remove intermeidate copy from xor chunk 2016-11-20 16:24:46 +01:00
Fabian Reinartz 7874d28f32 chunks: implement xor encoding 2016-11-20 14:33:00 +01:00
Fabian Reinartz 342aa82505 Fix pages and index package 2016-11-15 15:53:48 +01:00
Fabian Reinartz af36c89178 Make adapter work + comment it out
The adapter drags in a whole dependency chain of Prometheus internal
packages due to its storage's hideous structure.
With dependency flattening the adapter can actually be included into
Prometheus in a working way but it breaks package builds. Thus we
comment it out until we found a way to fix this contraption.
2016-11-15 11:46:17 +01:00
Fabian Reinartz 3ad4db56aa Remove Prometheus internal metric type from interface 2016-11-15 11:43:34 +01:00
Fabian Reinartz 40c27a1c2f Add benchmark utility 2016-11-15 10:35:42 +01:00
Fabian Reinartz cca6de1030 Add Prometheus adapter 2016-11-15 10:35:02 +01:00
Fabian Reinartz f9f11c4a08 Add main db implementation 2016-11-15 10:34:25 +01:00
Fabian Reinartz e79b5b38f4 Add pages package 2016-11-15 10:34:01 +01:00
Fabian Reinartz 752c7eea35 Add index package 2016-11-15 10:33:44 +01:00
Fabian Reinartz ae7b18ae34 Add chunk package 2016-11-15 10:33:34 +01:00
Fabian Reinartz 7771cdb519 Initial commit 2016-11-15 09:17:15 +01:00