Commit graph

15 commits

Author SHA1 Message Date
Fabian Reinartz 96d7f540d4 Persist series without allocating the full set
Change index persistence for series to not be accumulated in memory
before being written as one large batch. `Labels` and `ChunkMeta`
objects are reused.
This cuts down memory spikes during compaction of multiple blocks
significantly.

As part of the the Index{Reader,Writer} now have an explicit notion of
symbols and series must be inserted in order.
2017-08-06 12:06:41 +02:00
Joe Handzik e6cb86c029 Fix typo in index.md
Minor change, monotinically -> monotonically.

Signed-Off-By: Joe Handzik <joseph.t.handzik@hpe.com>
2017-07-24 15:23:29 -05:00
Goutham Veeramachaneni 44e9ae38b5
Incorporate PR feedback.
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-26 21:26:31 +05:30
Goutham Veeramachaneni bacb143b7e
Simplify tombstone and WAL Delete formats.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-24 13:42:56 +05:30
Goutham Veeramachaneni 9d388941ef
Add checksums and magic number to the tombstones.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-23 17:27:45 +05:30
Fabian Reinartz 108dc97b95 Documentation fixes 2017-05-02 14:16:44 +02:00
Fabian Reinartz 2032a11d98 Add padding between fixed-sized index sections 2017-05-02 12:43:51 +02:00
Fabian Reinartz 34ba92eeeb Move CRC back to chunks file, alignment for fixed-sized ints 2017-04-30 10:18:07 +02:00
Fabian Reinartz a54f46d5e7 Migrate last IndexWriter pieces to decbuf 2017-04-30 10:18:07 +02:00
Fabian Reinartz 35b62f001e Change offset table layout, add TOC, ... 2017-04-30 10:18:07 +02:00
Fabian Reinartz 9b4eafcc4c Simplify and document postings serialization 2017-04-30 10:10:18 +02:00
Fabian Reinartz 0aad526d1a Simplify label value index
This removes the flag from the label value index and serializes it using
encbufs.
TODO: move CRC32 checksum into label value index hash table for
referntial integrity.
2017-04-30 10:10:18 +02:00
Fabian Reinartz 433e73f865 Change series and symbol table format 2017-04-30 10:10:18 +02:00
Fabian Reinartz db4272b849 docs: add index file format 2017-04-30 10:10:18 +02:00
Fabian Reinartz d8b91b969c docs: add chunk file format 2017-04-30 10:10:18 +02:00