Commit graph

829 commits

Author SHA1 Message Date
Fabian Reinartz 8e1f97fad4 wal: add write ahead log package
This adds a new WAL that's agnostic to the actual record contents.
It's much simpler and should be more resilient than the existing one.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:24:40 -04:00
Fabian Reinartz 99a2c4314f
Merge pull request #330 from codwu/tsdb-delete
add rwmutex to prevent concurrent map read when delete series
2018-07-11 13:21:26 +02:00
codwu bc6ef0b94e rename mts to intvlGroups
Signed-off-by: codwu <wuhan9087@163.com>
2018-07-10 21:24:13 +08:00
codwu e4444ca48c update addInterval function and test.
Signed-off-by: codwu <wuhan9087@163.com>
2018-07-06 20:30:27 +08:00
codwu 667e539a7a Merge branch 'master' of https://github.com/prometheus/tsdb into tsdb-delete 2018-07-06 20:21:32 +08:00
Fabian Reinartz 77db94c07e
Merge pull request #348 from BenoitKnecht/fix-block-boundaries
Make sure blocks don't overlap to avoid outsider chunks
2018-07-05 12:54:34 +02:00
Benoît Knecht 24b223c161 db: add test for Querier returning too many blocks
Due to the way blocks used to overlap by 1 millisecond (see #347), when
requesting a 2-hour interval starting at `blocks[1].MinTime`, the
`Querier` would consider three blocks: `blocks[0]`, `blocks[1]` and
`blocks[2]`, because `blocks[0].MaxTime` and `blocks[2].MinTime` were in
that interval.

However, if the blocks don't overlap, only two blocks should be
returned: `blocks[1]` and `blocks[2]`. This test ensures that it's
indeed the case.

Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:35:21 +02:00
Benoît Knecht 1e1b2e163d Make interval overlap comparisons more explicit
Blocks are half-open intervals [a, b), while all other intervals
(chunks, head, ...) are closed intervals [a, b].

Make that distinction explicit by defining `OverlapsClosedInterval()`
methods for blocks and chunks, and using them in place of the more
generic `intervalOverlap()` function.

This change also fixes `db.Querier()` and `db.Delete()`, which could
previously return one extraneous block at the end of the specified
interval.

Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:35:08 +02:00
Benoît Knecht 4ed6b9ed72 db: add test for chunks that span beyond a block's boundaries
Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:32:05 +02:00
Benoît Knecht 0e4be5226a db: block MaxTime should not be part of the block
Block intervals are bound by `block.MinTime`, `block.MaxTime`, but they
define a half-open interval: `[block.MinTime, block.MaxTime).

However, when deciding if a chunk was part of a block or not, the
`intervalOverlap()` function would consider both the chunk and the block
intervals as being closed.

Rather than modify the login in `intervalOverlap()`, we explicitly
remove the last value from the interval when reading from head to
persist blocks.

Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:32:05 +02:00
Fabian Reinartz f87d00d78d
Merge pull request #356 from prometheus/logrollback
Log series on rollback
2018-06-28 15:12:52 +02:00
Fabian Reinartz ea607b9fc3 Log series on rollback
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-28 09:04:07 -04:00
Fabian Reinartz 16727277e4
Merge pull request #354 from prometheus/cleandelete
Add resilience to crashes during deletion
2018-06-28 11:57:00 +02:00
Fabian Reinartz 087c4c6d3a Update doc comment
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-28 05:55:01 -04:00
Fabian Reinartz d907928e57 Clarify docs, error on unexpected meta read errors
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-27 12:05:21 -04:00
Fabian Reinartz af9003dcef Add resilience to crashes during deletion
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-27 09:50:31 -04:00
Fabian Reinartz 0b200798fe
Merge pull request #344 from simonpasquier/fix-out-of-range-index
chunks: fix potential "index out of range" error
2018-06-27 10:48:29 +02:00
Fabian Reinartz 0778d80ccf
Merge pull request #353 from cpatulea/patch-2
Update Appender.Add comment for uint64 refs.
2018-06-27 10:30:38 +02:00
Catalin Patulea 40766622ee
Update Appender.Add comment for uint64 refs.
Follow-up to prometheus 0efecea6d4.

Signed-off-by: Catalin Patulea <catalinp@google.com>
2018-06-26 14:15:58 -04:00
codwu cd145c90d5 remove put function and use RLock in Iter function
Signed-off-by: codwu <wuhan9087@163.com>
2018-06-25 21:52:11 +08:00
Simon Pasquier f55ccd4ecb Add unit tests
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-25 11:25:22 +02:00
Simon Pasquier 22061306aa Address Julius's comment
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-20 09:19:49 +02:00
Simon Pasquier e8fc6c8774 index: fix another Uvarint() return check
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-08 14:42:24 +02:00
codwu 84a45cb79a add rwmutex to prevent concurrent map read when delete series
Signed-off-by: codwu <wuhan9087@163.com>
2018-06-08 19:52:01 +08:00
Simon Pasquier ee5fe8ea9f chunks: fix potential "index out of range" error
When binary.Uvarint() fails, the returned number of bytes is less than
or equal to zero.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-08 10:28:27 +02:00
Fabian Reinartz c848349f07
Merge pull request #341 from codesome/delete-new-blocks-on-error
Cleanup new blocks on 'CleanTombstones' faliure
2018-06-05 11:24:13 +02:00
Fabian Reinartz 9a96bc46de
Merge pull request #342 from prometheus/superq/promcon
Add link to PromCon 2017 video.
2018-06-05 10:10:43 +02:00
Ganesh Vernekar 0c93850cd5 Merge pull request #1 from krasi-georgiev/pr/341
Add Test for Tombstone Cleaning after a failure
2018-06-04 17:20:15 -04:00
Krasi Georgiev 6094f35aa2 simplify if-else,test before the tombstone failure, more comments
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-06-04 20:18:44 +01:00
Krasi Georgiev f31a0d6457 add Test for Tombstone Cleaning after a failure
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-06-04 18:34:30 +01:00
Ben Kochie d5250f9673
Add link to PromCon 2017 video.
Signed-off-by: Ben Kochie <superq@gmail.com>
2018-06-04 14:41:43 +02:00
Ganesh Vernekar 528439aa93 Cleanup new blocks on 'CleanTombstones' faliure.
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-06-01 15:48:38 -04:00
Simon Pasquier eab78875fe repair: improve error message
This change adds the directory path to the error message in case the
repair process fails.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-05-30 10:23:27 +02:00
Fabian Reinartz ae33d7873d
Merge pull request #337 from prometheus/flock
Move to flock lockfile
2018-05-29 15:14:13 -04:00
Fabian Reinartz 7b578dea32 Move to flock lockfile
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-05-29 14:35:48 -04:00
Fabian Reinartz 3cf059159e
Merge pull request #335 from prometheus/blockorder
Ensure correct block order on reload
2018-05-28 16:41:47 -04:00
Fabian Reinartz 76c1b2cdb6 Ensure correct block order on reload
Due to a regression blocks were no longer ordered by time before
being stored in memory. This made data intermittently become unavailable
for queries.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-05-28 16:00:36 -04:00
Krasi Georgiev 2aae939d6c
Merge pull request #256 from mattbostock/fix_bench_default_file
bench: Fix path to default sample file
2018-05-15 10:25:56 +03:00
Goutham Veeramachaneni f5a7e43384
Merge pull request #325 from FUSAKLA/fus-cmd-human-readable-timestamps
feat cmd/tsdb: added human readable print for timestamps on block list
2018-05-10 10:40:14 +02:00
Martin Chodur 30ca799ca7
feat cmd/tsdb: added human readable print for timestamps on block list 2018-05-08 13:35:06 +02:00
Goutham Veeramachaneni d298c5a939
Merge pull request #308 from oiooj/pr-fix-test
Fix labels bench test
2018-05-07 16:16:15 +02:00
Goutham Veeramachaneni 24a9f278d4
Merge pull request #323 from AlekSi/small-fixes
Small fixes
2018-05-07 16:15:44 +02:00
Alexey Palazhchenko 9283c68e7d Fix ineffective assignment.
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2018-05-07 16:02:49 +03:00
Alexey Palazhchenko 64cce683cc gofmt -s.
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2018-05-07 15:39:54 +03:00
Goutham Veeramachaneni 900eb53ee7
Merge pull request #321 from AlekSi/patch-1
Update README, add badges
2018-05-07 12:43:13 +02:00
Goutham Veeramachaneni d47357b95d
Merge pull request #322 from AlekSi/patch-2
Test with Go 1.10.x too.
2018-05-07 12:42:19 +02:00
Alexey Palazhchenko 19a47b620d Test with Go 1.10.x too.
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2018-05-07 12:12:58 +03:00
Alexey Palazhchenko 3644792640 Update README, add badges.
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2018-05-07 12:10:31 +03:00
Matt Bostock def6e5a574 db: Match comment case to function name (#316)
`validateBlockSequence` is not exported, make the letter case in the comment match the function name.

Signed-off-by: Matt Bostock <matt@mattbostock.com>
2018-04-13 16:50:15 +01:00
Mario Trangoni b7173eb0e5 fix some comments typos (#315) 2018-04-08 10:28:30 +01:00