Commit graph

263 commits

Author SHA1 Message Date
Goutham Veeramachaneni 239f8b9eb5
Modify tests to check for mint, maxt
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-04-14 00:37:21 +05:30
Goutham Veeramachaneni b60c2068bc
Make blockQuerier return data in valid time-range
Fixes #43

Added mint, maxt to chunkSeriesIterator. Adding a field there is
inevitable as something similar is required for ignoring deleted
time-ranges.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-04-14 00:36:14 +05:30
Goutham Veeramachaneni b9868c9f0b
Initial test for blockQuerier
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-04-13 19:57:31 +05:30
Goutham Veeramachaneni 0c5c4f07c5
Test "Stitching" SeriesIterators
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-04-09 19:45:22 +05:30
Goutham Veeramachaneni f8a8b77976
postings: Fix bad test and revert changed logic
A bad test was introduced a couple of commits ago which also made
changes to pass the bad test. Fixed the test and revert the changes.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-04-09 17:20:39 +05:30
Goutham Veeramachaneni 86929ed090
Added Test For indexReader.LabelValues
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-04-08 21:12:04 +05:30
Goutham Veeramachaneni 329dc1fa6c
Add Tests For mergedPostings.Seek
* Renamed mergedPostings to make it consistent across the repo
* Fixed bug with Seek
* Added Tests for Seek

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-04-08 20:42:29 +05:30
Fabian Reinartz c73a397da2 Adjust maximum samples per chunk. 2017-04-07 10:58:37 +02:00
Fabian Reinartz 7de2217011 Add fast-path for equality matching 2017-04-05 15:37:48 +02:00
Fabian Reinartz 1579e12011 Add cross-platform mmap 2017-04-04 16:54:48 +02:00
Fabian Reinartz 10c7c9acbe Adjust import names to new repository organisation 2017-04-04 11:27:26 +02:00
Fabian Reinartz 767b88a3a5 Parallelize commits to prevent deadlocks
This parallelizes commits to prevent deadlocks across inconsistently
locked heads. As commits are currently not fully atomic across
heads, this does decrease our guarantees.
2017-04-04 11:21:19 +02:00
Fabian Reinartz 1afd8080f7 Merge pull request #27 from Gouthamve/master
Add mockIndex And Refactor Tests To Use That
2017-03-31 13:53:02 +02:00
Goutham Veeramachaneni 71e05a22c7
Add mockIndex And Refactor Tests To Use That 2017-03-30 04:48:41 +05:30
Fabian Reinartz fda8969f51 Merge pull request #23 from Gouthamve/18-bytes-iterator
Implement Postings Iterator Over Bytes
2017-03-29 13:17:05 +02:00
Fabian Reinartz 87d48bf9de Merge branch 'master' of github.com:fabxc/tsdb 2017-03-27 19:07:27 +02:00
Fabian Reinartz a52980e0a8 Add workaround for deadlocks
This adds a workaround to avoid deadlocks for inconsistent write lock
order across headBlocks.
Things keep working if transactions only append data for the same
timestamp, which is generally the case for Prometheus.

Full behavior should be restored in a subsequent change.
2017-03-27 19:05:34 +02:00
Goutham Veeramachaneni 141499ff19
Add Tests For bigEndianPostings 2017-03-27 15:46:55 +05:30
Goutham Veeramachaneni 7b94a4e17d
Rename bytePostings To bigEndianPostings
* To be more specific about the contents of the byte slice.
2017-03-27 14:04:42 +05:30
Fabian Reinartz 6a87e1a926 Merge pull request #22 from Gouthamve/master
Add Sample Back
2017-03-27 09:55:55 +02:00
Goutham Veeramachaneni efb0dfe1be
Implement Postings Iterator Over Bytes
Closes fabxc/tsdb#18
2017-03-26 23:40:12 +05:30
Goutham Veeramachaneni 61f866bb94
Add Sample Back
The compilation and tests are broken as head.go requires sample which
has been moved to another package while moving BufferedSeriesIterator.

Duplication seemed better compared to exposing sample from tsdbutil.
2017-03-26 23:22:58 +05:30
Fabian Reinartz 3be4ef94ce Move BufferedSeriesIterator in own package
This functionality is useful for a lot of clients but not relevant to
the TSDB's core features.
2017-03-24 13:23:32 +01:00
Fabian Reinartz f85d89abc0 Move BufferedSeriesIterator in own package
This functionality is useful for a lot of clients but not relevant to
the TSDB's core features.
2017-03-24 10:20:39 +01:00
Fabian Reinartz e478d0e3bc Actually close olds blocks in reloadBlocks
This fixes a bug leaking memory because blocks were not actually closed
as the closing call references the initial, empty slice
2017-03-23 18:27:20 +01:00
Fabian Reinartz 70909ca8ad Ensure GC runs after each compactor call
GC is triggered rarely, which may cause unnecessarily high memory
spikes when running several compaction cycles in a row. Explicitly run
GC so we don't have idle bytes marked as used from the previous cycle.
2017-03-21 12:21:02 +01:00
Fabian Reinartz 789e8224ff Fix wrong comparison in head block resorting 2017-03-21 12:12:33 +01:00
Fabian Reinartz 55ee4b5b3b Merge branch 'master' of github.com:fabxc/tsdb 2017-03-21 10:11:39 +01:00
Fabian Reinartz c18e055d7c Fix races and add comments on remaining ones 2017-03-21 10:11:23 +01:00
Fabian Reinartz d3669bd8b1 Merge pull request #15 from Gouthamve/lint-vet
Lint and Vet Fixes
2017-03-21 09:58:45 +01:00
Fabian Reinartz a4be181d3c Merge branch 'master' into lint-vet 2017-03-21 09:58:34 +01:00
Fabian Reinartz e837034360 Merge pull request #14 from Gouthamve/log-update
Update kit/log To New API
2017-03-21 09:56:32 +01:00
Fabian Reinartz 9c93f8f2aa Fix various races
This fixes different race condition encoutnered when running Prometheus.
It reduces the overall performance in the synthetic benchmark a fair bit
but has no indiciations of impacting a real-world setup notably.
2017-03-20 14:45:27 +01:00
Fabian Reinartz 2ef3682560 Hotfix erroneous "label index missing" error 2017-03-20 11:37:06 +01:00
Fabian Reinartz 3635569257 Trigger reload correctly on interrupted compaction 2017-03-20 10:41:43 +01:00
Fabian Reinartz 2c999836fb Add Queryable interface to Block
This adds the Queryable interface to the Block interface. Head and
persisted blocks now implement their own Querier() method and thus
isolate customization (e.g. remapPostings) more cleanly.
2017-03-20 10:21:21 +01:00
Fabian Reinartz 11be2cc585 Add composed Block interfaces, remove head generation
This adds more lower-leve interfaces which are used to compose
to different Block interfaces.
The DB only uses interfaces instead of explicit persistedBlock and
headBlock. The headBlock generation property is dropped as the use-case
can be implemented using block sequence numbers.
2017-03-20 09:02:36 +01:00
Goutham Veeramachaneni 761e4768f3
Lint and Vet Fixes 2017-03-19 21:35:01 +05:30
Goutham Veeramachaneni df7db4ac07
Update kit/log To New API
NewContext has been removed couple of weeks back.
Ref: https://github.com/go-kit/kit/releases/tag/v0.4.0
2017-03-19 21:03:09 +05:30
Fabian Reinartz 303a4ec3bc Sync directory on retention delation 2017-03-19 13:50:35 +01:00
Fabian Reinartz 863d38dfee Fix unreturned lock 2017-03-17 15:56:19 +01:00
Fabian Reinartz d0770302ed Add retention deletion 2017-03-17 15:30:05 +01:00
Fabian Reinartz 4962175218 Fix deadlock between heads and headmtx
With hundreds of concurrent appenders the locking order between the
headBlocks on instantiating appenders and write locking the headmtx
is hard to impossible to get consistent.
Just never instantiate appenders while holding the headmtx lock in any
way.
2017-03-17 14:10:18 +01:00
Fabian Reinartz 65b846ae5b Remove unreturned locks, detect writes on closed heads 2017-03-17 12:12:50 +01:00
Fabian Reinartz e0b33a7a28 Remove leftover signaling channels from headBlock 2017-03-17 10:16:55 +01:00
Fabian Reinartz 51ea34d381 Fix label sorting benchmark 2017-03-17 10:07:10 +01:00
Fabian Reinartz 9a9e46611e Merge branch 'master' of github.com:fabxc/tsdb 2017-03-15 14:46:01 +01:00
Fabian Reinartz 34efe4e2c8 Drop position mapper from head block
The position mapper was intended to pre-computed "expensive" ordering
of label sets. It was expensive to update and caused a lot of trouble.
Skipping this optimization entirely actually revelead it was pointless
and even harmful from the e2e perspective.
2017-03-15 14:44:29 +01:00
Fabian Reinartz ad5812d83a Handle WAL corruption by truncating
This adds handling for various corruption scenarios of the WAL.
If corruption is encountered, we truncate the WAL after the last valid
entry transparently and continue appending after the offset.
2017-03-15 11:16:16 +01:00
Fabian Reinartz 04bb006e2b Handle WAL corruption by truncating
This adds handling for various corruption scenarios of the WAL.
If corruption is encountered, we truncate the WAL after the last valid
entry transparently and continue appending after the offset.
2017-03-14 19:30:23 +01:00