Commit graph

585 commits

Author SHA1 Message Date
Goutham Veeramachaneni 2fa647f50b Fix missing postings in Merge and Intersect (#77)
* Test for a previous implematation of Intersect

Before we were moving the postings list everytime we create a new
chained `intersectPostings`. That was causing some postings to be
skipped. This test fails on the older version.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>

* Advance on Seek only when valid.

Issue:
Before in mergedPostings and others we advance everytime we `Seek`,
which causes issues with `Intersect`.

Take the case, where we have a mergedPostings = m merging, a: {10, 20, 30} and
b: {15, 25, 35}. Everytime we `Seek`, we do a.Seek and b.Seek.

Now if we Intersect m with {21, 22, 23, 30}, we would do Seek({21,22,23}) which
would advance a and b beyond 30.

Fix:
Now we advance only when the seeking value is greater than the current
value, as the definition specifies.

Also, posting 0 will not be a valid posting and will be used to signal
finished or un-initialized PostingsList.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>

* Add test for Merge+Intersect edgecase.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>

* Add comments to trivial tests.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-12 09:44:41 +02:00
Fabian Reinartz 968821e575 Merge pull request #57 from telendt/posings_fix
Fix various postings implementations
2017-05-11 14:11:08 +02:00
Fabian Reinartz 291137781b Merge branch 'panic-fix2' of https://github.com/Gouthamve/tsdb into Gouthamve-panic-fix2 2017-05-09 16:22:19 +02:00
Fabian Reinartz 44769c1654 Allow disabling lock file 2017-05-09 12:52:47 +02:00
Fabian Reinartz 09cd2021de Merge pull request #75 from Gouthamve/head-gen
E2E test for headBlock
2017-05-05 18:56:53 +02:00
Goutham Veeramachaneni 8096d11e4e
Add bounds check to headBlockAppender
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-05 19:52:11 +05:30
Goutham Veeramachaneni c1939b7136
Simply loop away from using tracking variables.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-05 19:34:59 +05:30
Goutham Veeramachaneni 5d2e72269b
Simplfied loops and functions
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-05 19:22:07 +05:30
Goutham Veeramachaneni 0908b0d27e
Add an end-to-end test for headBlock
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-03 22:50:34 +05:30
Goutham Veeramachaneni 7bbbd55aad
Fix bug where having one chunk can cause panic
When we have only one chunk that is out of range, then we are returning
it unpopulated (w/o calling `Chunk(ref)`). This would cause a panic
downstream.

Fixes: prometheus/prometheus#2629

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-03 22:45:28 +05:30
Fabian Reinartz 77c937b8e1 Merge pull request #73 from Gouthamve/skip-duplicate
Handle duplicate & out of order values in same txn
2017-05-03 15:28:32 +02:00
Goutham Veeramachaneni 0becfc8eb7
Simplify checking by using readSeriesSet
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-03 16:04:50 +05:30
Fabian Reinartz 3b5d60d134 Merge pull request #74 from prometheus/more-license-info
Also add original license info to bstream.go
2017-05-03 06:46:28 +02:00
Julius Volz 0e2f8917ed Also add original license info to bstream.go
Plus, extend the messages a bit to mention the origin repo.
2017-05-03 01:32:09 +02:00
Goutham Veeramachaneni adaf4d2099
Handle duplicate & out of order values in same txn
Add docs about not erroring out on exact dupes.
Moved tests to require.*

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-03 02:36:40 +05:30
Fabian Reinartz 63d431b905 Merge pull request #61 from prometheus/checksum
IndexWriter improvements
2017-05-02 14:21:00 +02:00
Fabian Reinartz 108dc97b95 Documentation fixes 2017-05-02 14:16:44 +02:00
Fabian Reinartz 2032a11d98 Add padding between fixed-sized index sections 2017-05-02 12:43:51 +02:00
Fabian Reinartz 7778e92c78 Merge pull request #63 from prometheus/fabxc-patch-1
Add dgryski/go-tsz LICENSE to header
2017-05-02 11:27:52 +02:00
Fabian Reinartz 87676f4a26 Add dgryski/go-tsz LICENSE to header 2017-05-02 11:26:01 +02:00
Fabian Reinartz 085991c9da Merge pull request #71 from Gouthamve/tests
Fix #59
2017-05-01 11:46:16 +02:00
Goutham Veeramachaneni 8b43b0d2c1
Fix broken tests
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-01 15:01:17 +05:30
Goutham Veeramachaneni 6169c33fb8
Fix #59
Mutating the chunks can change their length. Hence referencing using
previous indices can cause panics.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-01 14:33:56 +05:30
Fabian Reinartz 34ba92eeeb Move CRC back to chunks file, alignment for fixed-sized ints 2017-04-30 10:18:07 +02:00
Fabian Reinartz a54f46d5e7 Migrate last IndexWriter pieces to decbuf 2017-04-30 10:18:07 +02:00
Fabian Reinartz 94f3fd9812 Move encoding helpers into separate file 2017-04-30 10:18:07 +02:00
Fabian Reinartz 35b62f001e Change offset table layout, add TOC, ... 2017-04-30 10:18:07 +02:00
Fabian Reinartz 8b1f514a2d index: validate current write stages 2017-04-30 10:18:07 +02:00
Fabian Reinartz 9b4eafcc4c Simplify and document postings serialization 2017-04-30 10:10:18 +02:00
Fabian Reinartz 0aad526d1a Simplify label value index
This removes the flag from the label value index and serializes it using
encbufs.
TODO: move CRC32 checksum into label value index hash table for
referntial integrity.
2017-04-30 10:10:18 +02:00
Fabian Reinartz d30b181406 Switch series serialization to use encbufs 2017-04-30 10:10:18 +02:00
Fabian Reinartz 2ebaf1af4f Add encode buffer and simplify symbol serialization 2017-04-30 10:10:18 +02:00
Fabian Reinartz 433e73f865 Change series and symbol table format 2017-04-30 10:10:18 +02:00
Fabian Reinartz da068500c6 Swap len and encoding fields in chunk entry 2017-04-30 10:10:18 +02:00
Fabian Reinartz df96d97dab Move chunk checksum 2017-04-30 10:10:18 +02:00
Fabian Reinartz db4272b849 docs: add index file format 2017-04-30 10:10:18 +02:00
Fabian Reinartz d8b91b969c docs: add chunk file format 2017-04-30 10:10:18 +02:00
Fabian Reinartz cdfb604d89 Merge pull request #64 from prometheus/dbref
Fix and test ref handling in DB appender
2017-04-28 16:08:00 +02:00
Fabian Reinartz 5b3cb11f3d Merge pull request #65 from prometheus/comment-fixes
Minor comment fixes and additions.
2017-04-28 16:07:16 +02:00
Fabian Reinartz 2c46815adb Merge pull request #66 from prometheus/fix-meta-error-handling
Fix closing of meta file if meta encoding failed
2017-04-28 16:06:42 +02:00
Fabian Reinartz b07a87768a Merge pull request #67 from prometheus/remove-skiplist
Remove unused skiplist types
2017-04-28 16:06:01 +02:00
Julius Volz 006f5d6b22 Merge pull request #68 from prometheus/simplify-equals
Simplify Labels.Equals()
2017-04-28 09:05:35 -05:00
Fabian Reinartz b098c36d27 Merge pull request #69 from prometheus/fix-finalize-tail-errhandling
Add missing error handling for finalizeTail() call
2017-04-28 16:05:19 +02:00
Julius Volz a25f874ec4 Simplify Labels.Equals()
Also extend the Compare() tests to also test Labels.Equals().
2017-04-28 16:01:37 +02:00
Julius Volz 9a3d857d8b Add missing error handling for finalizeTail() call 2017-04-28 15:59:23 +02:00
Julius Volz e374db22bc Remove unused skiplist types 2017-04-28 15:57:01 +02:00
Julius Volz 94f1ac599e Fix closing of meta file if meta encoding failed 2017-04-28 15:45:30 +02:00
Julius Volz 8d1fb4fa01 Minor comment fixes and additions. 2017-04-28 15:41:42 +02:00
Fabian Reinartz 54a7b98cff Fix and test ref handling in DB appender 2017-04-28 15:24:28 +02:00
Tomasz Elendt d7ad9d2679 Fix various postings implementations
* newIntersectPostings and newMergePostings should not advance passed
  postings automatically.
* listPostings and bigEndianPostings should not panic with index out
  of range when At() is called before Next()/Seek().
* Seek() call should advance listPostings/bigEndianPostings.
2017-04-24 23:07:06 +02:00