Fabian Reinartz
8e1f97fad4
wal: add write ahead log package
...
This adds a new WAL that's agnostic to the actual record contents.
It's much simpler and should be more resilient than the existing one.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:24:40 -04:00
Fabian Reinartz
99a2c4314f
Merge pull request #330 from codwu/tsdb-delete
...
add rwmutex to prevent concurrent map read when delete series
2018-07-11 13:21:26 +02:00
codwu
bc6ef0b94e
rename mts
to intvlGroups
...
Signed-off-by: codwu <wuhan9087@163.com>
2018-07-10 21:24:13 +08:00
codwu
e4444ca48c
update addInterval
function and test.
...
Signed-off-by: codwu <wuhan9087@163.com>
2018-07-06 20:30:27 +08:00
codwu
667e539a7a
Merge branch 'master' of https://github.com/prometheus/tsdb into tsdb-delete
2018-07-06 20:21:32 +08:00
Fabian Reinartz
77db94c07e
Merge pull request #348 from BenoitKnecht/fix-block-boundaries
...
Make sure blocks don't overlap to avoid outsider chunks
2018-07-05 12:54:34 +02:00
Benoît Knecht
24b223c161
db: add test for Querier returning too many blocks
...
Due to the way blocks used to overlap by 1 millisecond (see #347 ), when
requesting a 2-hour interval starting at `blocks[1].MinTime`, the
`Querier` would consider three blocks: `blocks[0]`, `blocks[1]` and
`blocks[2]`, because `blocks[0].MaxTime` and `blocks[2].MinTime` were in
that interval.
However, if the blocks don't overlap, only two blocks should be
returned: `blocks[1]` and `blocks[2]`. This test ensures that it's
indeed the case.
Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:35:21 +02:00
Benoît Knecht
1e1b2e163d
Make interval overlap comparisons more explicit
...
Blocks are half-open intervals [a, b), while all other intervals
(chunks, head, ...) are closed intervals [a, b].
Make that distinction explicit by defining `OverlapsClosedInterval()`
methods for blocks and chunks, and using them in place of the more
generic `intervalOverlap()` function.
This change also fixes `db.Querier()` and `db.Delete()`, which could
previously return one extraneous block at the end of the specified
interval.
Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:35:08 +02:00
Benoît Knecht
4ed6b9ed72
db: add test for chunks that span beyond a block's boundaries
...
Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:32:05 +02:00
Benoît Knecht
0e4be5226a
db: block MaxTime should not be part of the block
...
Block intervals are bound by `block.MinTime`, `block.MaxTime`, but they
define a half-open interval: `[block.MinTime, block.MaxTime).
However, when deciding if a chunk was part of a block or not, the
`intervalOverlap()` function would consider both the chunk and the block
intervals as being closed.
Rather than modify the login in `intervalOverlap()`, we explicitly
remove the last value from the interval when reading from head to
persist blocks.
Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:32:05 +02:00
Fabian Reinartz
f87d00d78d
Merge pull request #356 from prometheus/logrollback
...
Log series on rollback
2018-06-28 15:12:52 +02:00
Fabian Reinartz
ea607b9fc3
Log series on rollback
...
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-28 09:04:07 -04:00
Fabian Reinartz
16727277e4
Merge pull request #354 from prometheus/cleandelete
...
Add resilience to crashes during deletion
2018-06-28 11:57:00 +02:00
Fabian Reinartz
087c4c6d3a
Update doc comment
...
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-28 05:55:01 -04:00
Fabian Reinartz
d907928e57
Clarify docs, error on unexpected meta read errors
...
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-27 12:05:21 -04:00
Fabian Reinartz
af9003dcef
Add resilience to crashes during deletion
...
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-27 09:50:31 -04:00
Fabian Reinartz
0b200798fe
Merge pull request #344 from simonpasquier/fix-out-of-range-index
...
chunks: fix potential "index out of range" error
2018-06-27 10:48:29 +02:00
Fabian Reinartz
0778d80ccf
Merge pull request #353 from cpatulea/patch-2
...
Update Appender.Add comment for uint64 refs.
2018-06-27 10:30:38 +02:00
Catalin Patulea
40766622ee
Update Appender.Add comment for uint64 refs.
...
Follow-up to prometheus 0efecea6d4
.
Signed-off-by: Catalin Patulea <catalinp@google.com>
2018-06-26 14:15:58 -04:00
codwu
cd145c90d5
remove put
function and use RLock in Iter
function
...
Signed-off-by: codwu <wuhan9087@163.com>
2018-06-25 21:52:11 +08:00
Simon Pasquier
f55ccd4ecb
Add unit tests
...
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-25 11:25:22 +02:00
Simon Pasquier
22061306aa
Address Julius's comment
...
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-20 09:19:49 +02:00
Simon Pasquier
e8fc6c8774
index: fix another Uvarint() return check
...
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-08 14:42:24 +02:00
codwu
84a45cb79a
add rwmutex to prevent concurrent map read when delete series
...
Signed-off-by: codwu <wuhan9087@163.com>
2018-06-08 19:52:01 +08:00
Simon Pasquier
ee5fe8ea9f
chunks: fix potential "index out of range" error
...
When binary.Uvarint() fails, the returned number of bytes is less than
or equal to zero.
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-08 10:28:27 +02:00
Fabian Reinartz
c848349f07
Merge pull request #341 from codesome/delete-new-blocks-on-error
...
Cleanup new blocks on 'CleanTombstones' faliure
2018-06-05 11:24:13 +02:00
Fabian Reinartz
9a96bc46de
Merge pull request #342 from prometheus/superq/promcon
...
Add link to PromCon 2017 video.
2018-06-05 10:10:43 +02:00
Ganesh Vernekar
0c93850cd5
Merge pull request #1 from krasi-georgiev/pr/341
...
Add Test for Tombstone Cleaning after a failure
2018-06-04 17:20:15 -04:00
Krasi Georgiev
6094f35aa2
simplify if-else,test before the tombstone failure, more comments
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-06-04 20:18:44 +01:00
Krasi Georgiev
f31a0d6457
add Test for Tombstone Cleaning after a failure
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-06-04 18:34:30 +01:00
Ben Kochie
d5250f9673
Add link to PromCon 2017 video.
...
Signed-off-by: Ben Kochie <superq@gmail.com>
2018-06-04 14:41:43 +02:00
Ganesh Vernekar
528439aa93
Cleanup new blocks on 'CleanTombstones' faliure.
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-06-01 15:48:38 -04:00
Simon Pasquier
eab78875fe
repair: improve error message
...
This change adds the directory path to the error message in case the
repair process fails.
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-05-30 10:23:27 +02:00
Fabian Reinartz
ae33d7873d
Merge pull request #337 from prometheus/flock
...
Move to flock lockfile
2018-05-29 15:14:13 -04:00
Fabian Reinartz
7b578dea32
Move to flock lockfile
...
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-05-29 14:35:48 -04:00
Fabian Reinartz
3cf059159e
Merge pull request #335 from prometheus/blockorder
...
Ensure correct block order on reload
2018-05-28 16:41:47 -04:00
Fabian Reinartz
76c1b2cdb6
Ensure correct block order on reload
...
Due to a regression blocks were no longer ordered by time before
being stored in memory. This made data intermittently become unavailable
for queries.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-05-28 16:00:36 -04:00
Krasi Georgiev
2aae939d6c
Merge pull request #256 from mattbostock/fix_bench_default_file
...
bench: Fix path to default sample file
2018-05-15 10:25:56 +03:00
Goutham Veeramachaneni
f5a7e43384
Merge pull request #325 from FUSAKLA/fus-cmd-human-readable-timestamps
...
feat cmd/tsdb: added human readable print for timestamps on block list
2018-05-10 10:40:14 +02:00
Martin Chodur
30ca799ca7
feat cmd/tsdb: added human readable print for timestamps on block list
2018-05-08 13:35:06 +02:00
Goutham Veeramachaneni
d298c5a939
Merge pull request #308 from oiooj/pr-fix-test
...
Fix labels bench test
2018-05-07 16:16:15 +02:00
Goutham Veeramachaneni
24a9f278d4
Merge pull request #323 from AlekSi/small-fixes
...
Small fixes
2018-05-07 16:15:44 +02:00
Alexey Palazhchenko
9283c68e7d
Fix ineffective assignment.
...
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2018-05-07 16:02:49 +03:00
Alexey Palazhchenko
64cce683cc
gofmt -s.
...
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2018-05-07 15:39:54 +03:00
Goutham Veeramachaneni
900eb53ee7
Merge pull request #321 from AlekSi/patch-1
...
Update README, add badges
2018-05-07 12:43:13 +02:00
Goutham Veeramachaneni
d47357b95d
Merge pull request #322 from AlekSi/patch-2
...
Test with Go 1.10.x too.
2018-05-07 12:42:19 +02:00
Alexey Palazhchenko
19a47b620d
Test with Go 1.10.x too.
...
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2018-05-07 12:12:58 +03:00
Alexey Palazhchenko
3644792640
Update README, add badges.
...
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2018-05-07 12:10:31 +03:00
Matt Bostock
def6e5a574
db: Match comment case to function name ( #316 )
...
`validateBlockSequence` is not exported, make the letter case in the comment match the function name.
Signed-off-by: Matt Bostock <matt@mattbostock.com>
2018-04-13 16:50:15 +01:00
Mario Trangoni
b7173eb0e5
fix some comments typos ( #315 )
2018-04-08 10:28:30 +01:00