Commit graph

584 commits

Author SHA1 Message Date
Fabian Reinartz 357e33bd1e Merge pull request #88 from Gouthamve/backups-1
Snapshot support
2017-06-06 20:35:47 +02:00
Goutham Veeramachaneni ff4ccb6eb0
Remove unnecessary error from ToggleCompaction fns
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 23:45:23 +05:30
Goutham Veeramachaneni 261cd9f393
Incorporate feedback.
Move back to {Enable, Disable}Compactions.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 20:23:20 +05:30
Goutham Veeramachaneni a110a64abd
Add full Snapshot support
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 18:15:54 +05:30
Goutham Veeramachaneni a1c8425357
Initial implementation of HeadBlock Snapshots
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-05 13:48:31 +05:30
Goutham Veeramachaneni 3a5ae6b1a4 Merge pull request #82 from Gouthamve/deletes-1
Implement deletions
2017-05-27 22:06:45 +05:30
Goutham Veeramachaneni 29c73f05f2
Make sure that mint and maxt are not modified.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-27 21:59:49 +05:30
Goutham Veeramachaneni 44e9ae38b5
Incorporate PR feedback.
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-26 21:26:31 +05:30
Fabian Reinartz 2a940a591a Merge pull request #85 from prometheus/metrics
Add and fix metrics
2017-05-26 16:24:23 +02:00
Fabian Reinartz b3b9245ad6 Add and fix metrics 2017-05-26 15:13:03 +02:00
Goutham Veeramachaneni 6febabeb28
Final delete fixes.
* Make sure no reads happen on the block when delete is in progress.
* Fix bugs in compaction.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-26 16:31:45 +05:30
Goutham Veeramachaneni c211ec4f49
Fix concurrent map access.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-24 16:58:04 +05:30
Goutham Veeramachaneni bacb143b7e
Simplify tombstone and WAL Delete formats.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-24 13:42:56 +05:30
Goutham Veeramachaneni f29fb62fba
Make TombstoneReader a Getter.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-24 11:24:24 +05:30
Goutham Veeramachaneni 9bf7aa9af1
Misc. fixes incorporating feedback.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-23 18:13:30 +05:30
Goutham Veeramachaneni 31cf939448
Add NumTombstones to BlockMeta.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-23 17:37:04 +05:30
Goutham Veeramachaneni 9d388941ef
Add checksums and magic number to the tombstones.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-23 17:27:45 +05:30
Goutham Veeramachaneni 3eb4119ab1
Make HeadBlock use WAL.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-23 16:15:16 +05:30
Goutham Veeramachaneni 244b73fce1
Rename for clarity and consistency.
Misc. changes for code cleanliness.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-22 16:42:36 +05:30
Goutham Veeramachaneni 7ccf9d0211
Add test on blockQuerier which uses tombstones.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-22 13:31:57 +05:30
Goutham Veeramachaneni 8434019ad9
Merge branch 'master' into deletes-1
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-22 12:58:38 +05:30
Fabian Reinartz 4f2eb2057e Merge pull request #83 from prometheus/refstr
Switch append refs to string
2017-05-22 08:49:09 +02:00
Goutham Veeramachaneni 662d8173fe
Make Appends after Delete visible.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-22 11:28:24 +05:30
Goutham Veeramachaneni 009dd2cde5
Add test for mergedTombstoneReader
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-22 10:31:50 +05:30
Goutham Veeramachaneni 0b70333ef6
Add tests for tombstones.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-21 23:20:05 +05:30
Goutham Veeramachaneni d32eb25662
Synchronise deletes and compactions.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-20 13:21:10 +05:30
Goutham Veeramachaneni 3dbb400bef
Initial implementation of Delete on DB.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-20 00:35:50 +05:30
Goutham Veeramachaneni d6bd64357b
Fix Delete on HeadBlock
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-19 22:54:29 +05:30
Goutham Veeramachaneni 45d3db4e9e
Use a *mapTombstoneReader instead of map
We need to recalculate the sorted ref list everytime we make a
Tombstones() call. This avoids that.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-19 11:56:37 +05:30
Fabian Reinartz 7e6961b4ac Fix import and var name 2017-05-18 17:30:52 +02:00
Fabian Reinartz debfe9b1e5 Simplify AddFast and avoid errgroup Commits in general 2017-05-18 17:17:56 +02:00
Fabian Reinartz fb5c5535fc Misc cleanup 2017-05-18 16:38:45 +02:00
Fabian Reinartz d2673d8659 Completely drop empty blocks 2017-05-18 16:12:17 +02:00
Fabian Reinartz 39df7e2bba Switch blocks to ULID directories, drop sequenc numbers 2017-05-18 16:09:30 +02:00
Fabian Reinartz 285bc07030 Switch append refs to string 2017-05-18 10:56:57 +02:00
Goutham Veeramachaneni 22c1b5b492
Make SeriesSets use tombstones.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-17 14:49:42 +05:30
Goutham Veeramachaneni 34a86af3c6
Move tombstones to their own thing.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-17 08:36:56 +05:30
Goutham Veeramachaneni cea3c88f17
Add Tombstones() method to Block.
Also add Seek() to TombstoneReader

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-16 19:48:28 +05:30
Goutham Veeramachaneni 3de55171d3
Add tests for tombstones and deletedIterator
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-16 12:43:33 +05:30
Goutham Veeramachaneni 4f1d857590
Implement Delete on HeadBlock
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-15 23:28:14 +05:30
Fabian Reinartz c8438cfc81 Add mutex profiling to benchmark 2017-05-14 11:51:56 +02:00
Goutham Veeramachaneni 5579efbd5b
Initial implentation of Deletes on persistedBlock
Very much a WIP

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-14 14:36:26 +05:30
Fabian Reinartz 56e6af99f9 Merge pull request #79 from prometheus/typerest
Various type improvements
2017-05-14 11:02:31 +02:00
Fabian Reinartz 3479092d55 Merge pull request #78 from Gouthamve/deletes-1
Make Select() reusable.
2017-05-14 09:38:37 +02:00
Fabian Reinartz 8b51b7e2be Make WAL for HeadBlock composeable. 2017-05-13 18:14:18 +02:00
Goutham Veeramachaneni d57f269eb4
Make Select() reusable.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-13 21:13:25 +05:30
Fabian Reinartz 4862b261d0 Abstract WAL into interface 2017-05-13 17:09:26 +02:00
Fabian Reinartz 535532ca02 Export refdSample
The type was part of a exported method signatures and should therefore
be exported as well.
2017-05-12 17:06:26 +02:00
Fabian Reinartz 5534e6c53c Make HeadBlock impl public, make interface private 2017-05-12 16:34:41 +02:00
Goutham Veeramachaneni 2fa647f50b Fix missing postings in Merge and Intersect (#77)
* Test for a previous implematation of Intersect

Before we were moving the postings list everytime we create a new
chained `intersectPostings`. That was causing some postings to be
skipped. This test fails on the older version.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>

* Advance on Seek only when valid.

Issue:
Before in mergedPostings and others we advance everytime we `Seek`,
which causes issues with `Intersect`.

Take the case, where we have a mergedPostings = m merging, a: {10, 20, 30} and
b: {15, 25, 35}. Everytime we `Seek`, we do a.Seek and b.Seek.

Now if we Intersect m with {21, 22, 23, 30}, we would do Seek({21,22,23}) which
would advance a and b beyond 30.

Fix:
Now we advance only when the seeking value is greater than the current
value, as the definition specifies.

Also, posting 0 will not be a valid posting and will be used to signal
finished or un-initialized PostingsList.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>

* Add test for Merge+Intersect edgecase.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>

* Add comments to trivial tests.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-12 09:44:41 +02:00