Commit graph

701 commits

Author SHA1 Message Date
Bartek Plotka 51ce1cc7ff db: Fixed validateBlockSequence.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-28 18:33:41 +01:00
Bartek Plotka a9b28a6aa0 db: Added tests for validateBlockSequence to confirm a bug.
(That's why test fails)

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-28 15:50:52 +01:00
Goutham Veeramachaneni 195bc0d286
Merge pull request #303 from Bplotka/bp/better-compact-logging
repair + compact: Improved logging for easier future debug purposes.
2018-03-16 00:45:47 +05:30
Bartek Plotka fada85a83c repair + compact: Improved logging for easier future debug purposes.
This is based on my experience while debugging https://github.com/prometheus/prometheus/issues/3943.

I needed to deduct few things, and all that would be just bit easier with these two logs:
- new block's ULID on each compaction.
- actual list of Blocks (ulid + time range) on Prometheus startup (easy to log that while repairing blocks).

We don't really need blocks that takes part in compaction - that can be deducted easily based on time ranges of blocks we have currently in system.

What do you think?

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-15 14:59:32 +00:00
Fabian Reinartz 659ed64429
Merge pull request #302 from Gouthamve/fix-prom-3957
Add architecture specific map sizes
2018-03-14 09:49:50 -04:00
Goutham Veeramachaneni 6e5164fe44
Add architecture specific map sizes
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2018-03-14 18:59:20 +05:30
Goutham Veeramachaneni 902e1ffad4
Merge pull request #301 from Gouthamve/fix-prom-3957
Use the right sized byte array for large indexes
2018-03-14 18:17:08 +05:30
Goutham Veeramachaneni fb65e3daf4
Use the right sized byte array for large indexes
Fixes: https://github.com/prometheus/prometheus/issues/3957

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2018-03-14 16:47:37 +05:30
Fabian Reinartz 7a62f6a0f8
Merge pull request #300 from Gouthamve/fix-prom-3956
Close file for windows platform
2018-03-14 07:17:19 -04:00
Goutham Veeramachaneni 9744de20cc
Close file for windows platform
Windows needs files that are being deleted to be closed.

Fixes: https://github.com/prometheus/prometheus/issues/3956

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2018-03-14 13:08:40 +05:30
Fabian Reinartz 00404ae5ab
Merge pull request #299 from Bplotka/fix/issue3943
compact: Assume fresh block to ignore by minTime not ULID order. Added tests.
2018-03-13 21:20:03 +01:00
Bartek Plotka 2aceac0a12 Fixed some comments.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-13 15:30:00 +00:00
Bartek Plotka 328c0ff5b0 Simplified the flow and tests.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-13 14:12:32 +00:00
Bartek Plotka 483da43660 compact: Exclude last block with shortest range instead of newest one by ULID.
Fixes https://github.com/prometheus/prometheus/issues/3943 issue.

Added tests.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-03-13 12:30:27 +00:00
Fabian Reinartz 16b2bf1b45
Merge pull request #291 from mattbostock/patch-2
Fix typo in db.go
2018-03-02 12:51:49 +01:00
Matt Bostock aa517c39bb
Fix typo in db.go 2018-03-02 11:12:32 +00:00
Fabian Reinartz 66666f091a
Merge pull request #290 from prometheus/snap
Improve usability of snapshots
2018-02-28 12:25:42 +01:00
Fabian Reinartz 8cd32d8fd4 Improve usability of snapshots 2018-02-28 12:07:34 +01:00
Fabian Reinartz f6c76078f5
Merge pull request #288 from krasi-georgiev/tests-cleanup-tmps
Tests cleanup tmps
2018-02-26 15:03:14 +01:00
Krasi Georgiev eb6cdbcb5b some test funcitons didn't cleanup after themselves. 2018-02-23 18:45:59 +02:00
Fabian Reinartz 88ddcab17c Merge branch 'master' of github.com:prometheus/tsdb 2018-02-21 21:11:26 +01:00
Fabian Reinartz 3f06121dd0 Add Version getter for index reader 2018-02-21 21:06:19 +01:00
Matt Bostock bc97c6c6be Misc fixes (#285)
* Fix typo in head.go

pralellize -> paralellize

* Remove commented out code

It's dead code, remove it.

* Correct reference to sample buffer
2018-02-21 16:38:59 +01:00
Fabian Reinartz 494acd3070
Merge pull request #273 from Gouthamve/large-idx-reader
Fix reader for large index files.
2018-02-15 12:29:47 +01:00
Fabian Reinartz cc673dc167
Merge pull request #282 from codesome/goroutine-stack-overflow
Change recursion into iteration for removedPostings.Next()
2018-02-15 12:20:47 +01:00
Ganesh Vernekar 0fb73ed622 Changed recursion into iteration for removedPostings.Next() 2018-02-14 19:27:35 +05:30
Fabian Reinartz bc49a665d1
Merge pull request #281 from prometheus/indexversion
Indexversion
2018-02-12 11:58:46 +01:00
Fabian Reinartz a345231a0e Add test for block meta version and comments 2018-02-12 11:46:22 +01:00
Fabian Reinartz 04d5ac5aae Add repair test 2018-02-09 13:43:42 +01:00
Fabian Reinartz d09283f60a Add repair of bad index version 2018-02-09 13:11:03 +01:00
Fabian Reinartz 72d61bcf4e
Merge pull request #257 from mattbostock/remove_unused_field
Remove unused field from xorAppender
2018-02-09 12:30:10 +01:00
Fabian Reinartz 5b26bd5f6f
Merge pull request #252 from simonpasquier/add-cuttoff-metrics
Count the total & failed numbers of block cutoffs
2018-02-09 12:28:22 +01:00
Fabian Reinartz ffe73cd0a6
Merge pull request #246 from nipuntalukdar/fixatest
fixed an issue in TestWALRestoreCorrupted
2018-02-05 17:47:31 +01:00
Goutham Veeramachaneni 6c62437101 Fix metric unit (#275)
Fixes #274

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2018-02-05 08:36:18 +00:00
Goutham Veeramachaneni bb0e74b343
Fix reader for large index files.
Currently the offsets are cast into uint32 even though the index can
grow larger than 4GiB.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2018-02-05 13:51:22 +05:30
Goutham Veeramachaneni 44dd5e1202
Merge pull request #269 from Gouthamve/fix-reloads
Fix extra reloads.
2018-01-26 20:24:38 +05:30
Goutham Veeramachaneni b7bea04e62
Fix extra reloads.
Previously we reloaded every 1 minute irrespective of whether there are
deleted blocks or not.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2018-01-26 19:33:59 +05:30
Goutham Veeramachaneni ad0fdaf436
Merge pull request #265 from simonpasquier/fix-nil-index-writer
Fix panic on nil index writer
2018-01-24 20:28:35 +05:30
Simon Pasquier 5d47710f8a Fix panic on nil index writer 2018-01-24 15:55:04 +01:00
Goutham Veeramachaneni 467948f3c3
Merge pull request #262 from cstyan/callum-249
Reduce index file size caused by symbol references by using indicies rather than offsets
2018-01-18 14:02:54 +05:30
Callum Styan cd67584817 we don't want to write to index file in older version formats 2018-01-17 22:46:22 -08:00
Callum Styan 87a9415bf1 update docs for symbols indexing change 2018-01-17 14:00:06 -08:00
Fabian Reinartz c7e30f9211
Merge pull request #261 from Gouthamve/corrupt-patch
Add block ULID to index reader errors
2018-01-17 10:51:45 +01:00
Callum Styan 4792d2bbd1 handle v1 vs v2 for symbol offset vs index changes 2018-01-16 20:37:57 -08:00
Callum Styan a515b16299 use an incrementing index rather than an offset for symbols table 2018-01-16 20:07:59 -08:00
Goutham Veeramachaneni 2c3400ab4e Add block ULID to index reader errors
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2018-01-16 23:48:58 +05:30
Matt Bostock 2023d8a798 Remove unused field from xorAppender
This field is not used, remove it.
2018-01-13 20:57:12 +00:00
Matt Bostock 50211e89ad Fix typos in comments (#254)
a the -> the
timestmap -> timestamp
badded -> padded
its -> it is
callers -> caller's
2018-01-13 17:51:50 +00:00
Goutham Veeramachaneni 8d373c763b
Merge pull request #248 from shubheksha/fix/238
Align series to 8/16 byte padding to increase addressable space
2018-01-13 12:25:28 +05:30
Shubheksha Jalan 618bceadb4 move version check into newReader() 2018-01-13 00:36:20 +05:30