Commit graph

18 commits

Author SHA1 Message Date
Fabian Reinartz b7c3cfecbf index: abstract ByteSlice and adjust indexReader
This replaces the builtin byte slice with an interface for the index
reader. This allows the complex decoding of the index file format
to be used against more generalized implementations.
2017-11-09 17:38:32 +00:00
Fabian Reinartz fb9da52b11 Add more verbose error handling for closing, reduce locking
This commit introduces error returns in various places and is explicit
about closing persisted blocks.
{Index,Chunk,Tombstone}Readers are more consistent about their Close()
method. Whenever a reader is retrieved, the corresponding close method
must eventually be called. We use this to track pending readers against
persisted blocks.

Querier's against the DB no longer hold a read lock for their entire
lifecycle. This avoids long running queriers to starve new ones when we
have to acquire a write lock when reloading blocks.
2017-10-10 12:13:37 +02:00
Fabian Reinartz 78df406dac Allocate and cache strings for persisted blocks
This change loads the full symbol table when we open a persisted block
and allocates a string for each. This ensures that strings retrieved
through the index can be used after the block was closed.
Before we backed the strings by the mmap'd byte regions which would
segfault in this case.

Also remove an inconsistency in the disk format and move both offset
tables to the end (breaking change).
2017-10-02 15:56:57 +02:00
Fabian Reinartz c36d574290 Replace single head lock with granular locks
This adds various new locks to replace the single big lock on
the head. All parts now must be COW as they may be held by clients
after initial retrieval.
Series by ID and hashes are now held in a stripe lock to reduce
contention and total holding time during GC. This should reduce
starvation of readers.
2017-09-05 14:41:39 +02:00
Fabian Reinartz 1ddedf2b30 Change series ID from uint32 to uint64 2017-09-04 16:08:38 +02:00
Fabian Reinartz 2644c8665c Don't allocate ChunkMetas, reuse postings slices 2017-08-06 20:41:24 +02:00
Fabian Reinartz 96d7f540d4 Persist series without allocating the full set
Change index persistence for series to not be accumulated in memory
before being written as one large batch. `Labels` and `ChunkMeta`
objects are reused.
This cuts down memory spikes during compaction of multiple blocks
significantly.

As part of the the Index{Reader,Writer} now have an explicit notion of
symbols and series must be inserted in order.
2017-08-06 12:06:41 +02:00
Fabian Reinartz df96d97dab Move chunk checksum 2017-04-30 10:10:18 +02:00
Fabian Reinartz a5322c50fc Merge pull request #36 from Gouthamve/tests
Tests For Querier and Postings
2017-04-24 10:50:59 +02:00
Fabian Reinartz 778103b450 Add liecence file and headers 2017-04-10 20:59:45 +02:00
Goutham Veeramachaneni 86929ed090
Added Test For indexReader.LabelValues
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-04-08 21:12:04 +05:30
Fabian Reinartz 10c7c9acbe Adjust import names to new repository organisation 2017-04-04 11:27:26 +02:00
Goutham Veeramachaneni 71e05a22c7
Add mockIndex And Refactor Tests To Use That 2017-03-30 04:48:41 +05:30
Fabian Reinartz a8e8903350 Use ChunkMeta references for clarity
This has been a common source of hard to debug issues. Its a premature
and unbenchmarked optimization and semantically, we want ChunkMetas to
be references in all changed cases.
2017-03-14 15:40:16 +01:00
Fabian Reinartz d6fb6aaaa8 Rename paritionSeriesSet to mergedSeriesSet 2017-03-14 15:24:08 +01:00
Fabian Reinartz 5876a3303c Add test for Index postings querying 2017-03-09 09:39:30 +01:00
Fabian Reinartz ca1bc920b7 Test updates 2017-03-08 16:54:13 +01:00
Fabian Reinartz 8a7addfc44 Split persistence by chunk/index instead of read/write 2017-03-07 12:48:52 +01:00
Renamed from rw_test.go (Browse further)