Commit graph

257 commits

Author SHA1 Message Date
Krasi Georgiev d38516b1c2
remove unused changes variable (#391)
This was added in
55a9b5428a

and later not used after some refactoring in following PRs.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-09-21 09:24:01 +03:00
beorn7 3bc6c670fa Revert "Remove prometheus_ prefix from metrics"
This reverts commit 98fe30438c.

After some discussion, it was concluded that we want the full
`prometheus_tsdb_...` prefix hardcoded in the library.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-18 19:19:19 +02:00
Callum Styan 722f0ab920 break MigrateWAL into two functions, detection and migration (#371)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2018-09-17 19:30:56 +03:00
beorn7 98fe30438c Remove prometheus_ prefix from metrics
This can now be added by users of the library as needed with the new
https://godoc.org/github.com/prometheus/client_golang/prometheus#WrapRegistererWithPrefix

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-17 14:54:28 +02:00
Bob Shannon cb7f320d42 Expose prometheus_tsdb_lowest_timestamp metric (#363)
* Expose prometheus_tsdb_start_time_seconds metric

Signed-off-by: Bob Shannon <bshannon@palantir.com>

* Search for block with smallest minTime

Signed-off-by: Bob Shannon <bshannon@palantir.com>

* PR comments

Signed-off-by: Bob Shannon <bshannon@palantir.com>

* PR comment: Make metric name more accurate

Signed-off-by: Bob Shannon <bshannon@palantir.com>
2018-09-14 17:37:45 +05:30
Ganesh Vernekar 2945db18ca Changes in series names (and types) exposed (#376)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-12 14:39:02 +05:30
Ganesh Vernekar 77d2c7cf3d
Added metric for symbol table size
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-08 23:58:36 +05:30
Fabian Reinartz 1a5573b4ce Migrate write ahead log
On startup, rewrite the old write ahead log into the new format once.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:34:18 -04:00
Fabian Reinartz 7841d417b3 Ensure blocks are time-ordered in memory
We assume in multiple places that the block list held by DB
has blocks sequential by time.
A regression caused us to hold them ordered by ULID, i.e. by creation
time instead.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:25:30 -04:00
Fabian Reinartz def912ce0e Integrate new WAL and checkpoints
Remove the old WAL and drop in the new one

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:25:30 -04:00
Benoît Knecht 1e1b2e163d Make interval overlap comparisons more explicit
Blocks are half-open intervals [a, b), while all other intervals
(chunks, head, ...) are closed intervals [a, b].

Make that distinction explicit by defining `OverlapsClosedInterval()`
methods for blocks and chunks, and using them in place of the more
generic `intervalOverlap()` function.

This change also fixes `db.Querier()` and `db.Delete()`, which could
previously return one extraneous block at the end of the specified
interval.

Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:35:08 +02:00
Benoît Knecht 0e4be5226a db: block MaxTime should not be part of the block
Block intervals are bound by `block.MinTime`, `block.MaxTime`, but they
define a half-open interval: `[block.MinTime, block.MaxTime).

However, when deciding if a chunk was part of a block or not, the
`intervalOverlap()` function would consider both the chunk and the block
intervals as being closed.

Rather than modify the login in `intervalOverlap()`, we explicitly
remove the last value from the interval when reading from head to
persist blocks.

Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:32:05 +02:00
Fabian Reinartz 087c4c6d3a Update doc comment
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-28 05:55:01 -04:00
Fabian Reinartz d907928e57 Clarify docs, error on unexpected meta read errors
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-27 12:05:21 -04:00
Fabian Reinartz af9003dcef Add resilience to crashes during deletion
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-27 09:50:31 -04:00
Catalin Patulea 40766622ee
Update Appender.Add comment for uint64 refs.
Follow-up to prometheus 0efecea6d4.

Signed-off-by: Catalin Patulea <catalinp@google.com>
2018-06-26 14:15:58 -04:00
Krasi Georgiev 6094f35aa2 simplify if-else,test before the tombstone failure, more comments
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-06-04 20:18:44 +01:00
Ganesh Vernekar 528439aa93 Cleanup new blocks on 'CleanTombstones' faliure.
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-06-01 15:48:38 -04:00
Fabian Reinartz 7b578dea32 Move to flock lockfile
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-05-29 14:35:48 -04:00
Fabian Reinartz 76c1b2cdb6 Ensure correct block order on reload
Due to a regression blocks were no longer ordered by time before
being stored in memory. This made data intermittently become unavailable
for queries.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-05-28 16:00:36 -04:00
Matt Bostock def6e5a574 db: Match comment case to function name (#316)
`validateBlockSequence` is not exported, make the letter case in the comment match the function name.

Signed-off-by: Matt Bostock <matt@mattbostock.com>
2018-04-13 16:50:15 +01:00
Bartek Plotka 00594b85cd db: Addressed comments.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-04-05 16:53:24 +01:00
Bartek Plotka 03e94365e1 db: Made overlap String() prettier and more readable.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-04-05 16:01:16 +01:00
Fabian Reinartz 00e13f519a
Merge pull request #310 from Bplotka/overlap-detection-fix
db: Fixed validateBlockSequence, exported it and added tests.
2018-04-05 15:30:22 +02:00
Bartek Plotka 15b5d89222 db: Addressed comments.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-04-05 14:15:24 +01:00
Bartek Plotka cc306ef0d5 Added grouping by overlap range.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-04-05 13:51:33 +01:00
Bartek Plotka 7412e2b44b Added more cases and modified one var name.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-29 12:50:46 +01:00
Bartek Plotka f07d829946 db: Tiny tuning of algo + added proper print.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-28 23:50:42 +01:00
Bartek Plotka c8b4a7b839 db: Simplified algorithm.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-28 23:18:24 +01:00
Bartek Plotka 51ce1cc7ff db: Fixed validateBlockSequence.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-28 18:33:41 +01:00
Bartek Plotka a9b28a6aa0 db: Added tests for validateBlockSequence to confirm a bug.
(That's why test fails)

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-28 15:50:52 +01:00
Mario Trangoni e5dabad1d7 fix megacheck issue: simplified return err 2018-03-21 22:39:15 +01:00
Mario Trangoni 09142e4dd1 fix unconvert issues: unnecessary conversion 2018-03-21 22:39:14 +01:00
Matt Bostock aa517c39bb
Fix typo in db.go 2018-03-02 11:12:32 +00:00
Fabian Reinartz 8cd32d8fd4 Improve usability of snapshots 2018-02-28 12:07:34 +01:00
Fabian Reinartz a345231a0e Add test for block meta version and comments 2018-02-12 11:46:22 +01:00
Fabian Reinartz 04d5ac5aae Add repair test 2018-02-09 13:43:42 +01:00
Fabian Reinartz d09283f60a Add repair of bad index version 2018-02-09 13:11:03 +01:00
Fabian Reinartz 5b26bd5f6f
Merge pull request #252 from simonpasquier/add-cuttoff-metrics
Count the total & failed numbers of block cutoffs
2018-02-09 12:28:22 +01:00
Goutham Veeramachaneni b7bea04e62
Fix extra reloads.
Previously we reloaded every 1 minute irrespective of whether there are
deleted blocks or not.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2018-01-26 19:33:59 +05:30
Simon Pasquier e9fc37be61 Count the total and failed block cutoffs 2018-01-11 10:22:59 +01:00
Fabian Reinartz 7087f68647 Return new block ULID in compactor 2017-12-21 12:02:05 +01:00
Fabian Reinartz 67f0ca8f0e Move index and chunk encoders to own packages 2017-12-21 11:27:54 +01:00
Goutham Veeramachaneni 30bbbe34f8
Merge pull request #210 from Gouthamve/tomb-clean
Add a function to cleanup tombstones.
2017-11-30 15:28:01 +05:30
Goutham Veeramachaneni f35e9c7fec
Merge branch 'master' into fix-dels
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-11-30 13:30:13 +05:30
Goutham Veeramachaneni 8a5ea9db74
Merge branch 'master' into tomb-clean
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-11-30 13:15:23 +05:30
Goutham Veeramachaneni e7445d00b0
Add a function to cleanup tombstones.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-11-27 20:03:37 +05:30
Fabian Reinartz ad3c4849a9
Merge pull request #212 from prometheus/fabxc
Cleanup, small refactoring, expose a few things
2017-11-23 18:41:24 +01:00
Goutham Veeramachaneni 41fd9c66ef
Close the retention blocks before deleting them.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-11-23 18:57:10 +05:30
Goutham Veeramachaneni 1627a47640
A copy is being assigned newChunk not the original
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-11-22 17:58:06 +05:30