Commit graph

38 commits

Author SHA1 Message Date
Fabian Reinartz 96d7f540d4 Persist series without allocating the full set
Change index persistence for series to not be accumulated in memory
before being written as one large batch. `Labels` and `ChunkMeta`
objects are reused.
This cuts down memory spikes during compaction of multiple blocks
significantly.

As part of the the Index{Reader,Writer} now have an explicit notion of
symbols and series must be inserted in order.
2017-08-06 12:06:41 +02:00
Goutham Veeramachaneni a110a64abd
Add full Snapshot support
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 18:15:54 +05:30
Goutham Veeramachaneni 34a86af3c6
Move tombstones to their own thing.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-17 08:36:56 +05:30
Goutham Veeramachaneni 4f1d857590
Implement Delete on HeadBlock
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-15 23:28:14 +05:30
Goutham Veeramachaneni 5579efbd5b
Initial implentation of Deletes on persistedBlock
Very much a WIP

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-14 14:36:26 +05:30
Fabian Reinartz 2032a11d98 Add padding between fixed-sized index sections 2017-05-02 12:43:51 +02:00
Fabian Reinartz 34ba92eeeb Move CRC back to chunks file, alignment for fixed-sized ints 2017-04-30 10:18:07 +02:00
Fabian Reinartz a54f46d5e7 Migrate last IndexWriter pieces to decbuf 2017-04-30 10:18:07 +02:00
Fabian Reinartz 94f3fd9812 Move encoding helpers into separate file 2017-04-30 10:18:07 +02:00
Fabian Reinartz 35b62f001e Change offset table layout, add TOC, ... 2017-04-30 10:18:07 +02:00
Fabian Reinartz 8b1f514a2d index: validate current write stages 2017-04-30 10:18:07 +02:00
Fabian Reinartz 9b4eafcc4c Simplify and document postings serialization 2017-04-30 10:10:18 +02:00
Fabian Reinartz 0aad526d1a Simplify label value index
This removes the flag from the label value index and serializes it using
encbufs.
TODO: move CRC32 checksum into label value index hash table for
referntial integrity.
2017-04-30 10:10:18 +02:00
Fabian Reinartz d30b181406 Switch series serialization to use encbufs 2017-04-30 10:10:18 +02:00
Fabian Reinartz 2ebaf1af4f Add encode buffer and simplify symbol serialization 2017-04-30 10:10:18 +02:00
Fabian Reinartz 433e73f865 Change series and symbol table format 2017-04-30 10:10:18 +02:00
Fabian Reinartz df96d97dab Move chunk checksum 2017-04-30 10:10:18 +02:00
Julius Volz 8d1fb4fa01 Minor comment fixes and additions. 2017-04-28 15:41:42 +02:00
Fabian Reinartz 778103b450 Add liecence file and headers 2017-04-10 20:59:45 +02:00
Fabian Reinartz 7de2217011 Add fast-path for equality matching 2017-04-05 15:37:48 +02:00
Fabian Reinartz 10c7c9acbe Adjust import names to new repository organisation 2017-04-04 11:27:26 +02:00
Goutham Veeramachaneni 71e05a22c7
Add mockIndex And Refactor Tests To Use That 2017-03-30 04:48:41 +05:30
Goutham Veeramachaneni 7b94a4e17d
Rename bytePostings To bigEndianPostings
* To be more specific about the contents of the byte slice.
2017-03-27 14:04:42 +05:30
Goutham Veeramachaneni efb0dfe1be
Implement Postings Iterator Over Bytes
Closes fabxc/tsdb#18
2017-03-26 23:40:12 +05:30
Fabian Reinartz 2ef3682560 Hotfix erroneous "label index missing" error 2017-03-20 11:37:06 +01:00
Fabian Reinartz a8e8903350 Use ChunkMeta references for clarity
This has been a common source of hard to debug issues. Its a premature
and unbenchmarked optimization and semantically, we want ChunkMetas to
be references in all changed cases.
2017-03-14 15:40:16 +01:00
Fabian Reinartz 8a7addfc44 Split persistence by chunk/index instead of read/write 2017-03-07 12:48:52 +01:00
Fabian Reinartz 2a825f6c28 Consolidate mem index into HeadBlock 2016-12-22 01:12:28 +01:00
Fabian Reinartz aabb21f4b9 Add shard series set test 2016-12-19 11:44:11 +01:00
Fabian Reinartz d56b281006 Rename Iterator to Postings 2016-12-15 07:05:29 +01:00
Fabian Reinartz ca89080128 Misc fixes for initial Prometheus integration 2016-12-14 18:38:46 +01:00
Fabian Reinartz 6eeb0ef01c Add queriers and partial mocks 2016-12-13 15:26:58 +01:00
Fabian Reinartz 3a528c3078 Write plain postings list index 2016-12-10 09:44:00 +01:00
Fabian Reinartz 0cf8bb9e53 Move sub-indexes into single index structure 2016-12-09 10:41:51 +01:00
Fabian Reinartz 2c34a15fe6 Add initial seriailization of block data 2016-12-08 17:43:10 +01:00
Fabian Reinartz 83574b1565 Add new interfaces and skeleton 2016-12-04 13:16:11 +01:00
Fabian Reinartz 6f93a699e6 wipe 2016-12-02 17:49:05 +01:00
Fabian Reinartz f9f11c4a08 Add main db implementation 2016-11-15 10:34:25 +01:00