Krasi Georgiev
288f67efbf
fix some unchecked errors and remove unused vars. ( #592 )
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-04-25 13:07:04 +03:00
Krasi Georgiev
c3ffdf1a99
Test createBlock and check all os.RemoveAll in the tests for errors. ( #549 )
...
Testing that createBlock creates blocks that can be opened.
and checking the os.RemoveAll for errors will catch errors for un-closed files under windows.
Many missing .Close() calls were added for fixing failing os.RemoveAll
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-03-19 15:31:57 +02:00
Krasi Georgiev
48c439d26d
fix statick check errors ( #475 )
...
fix the tests for `check_license` and `staticcheck`
the static check also found some actual bugs.
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-02 19:48:42 +03:00
Krasi Georgiev
2962202ed3
fix windows tests ( #469 )
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-12-13 16:29:29 +03:00
Ben
a8351dc9d0
Using filepath.Join() instead of strings with slashes ( #428 )
...
fixes: https://github.com/prometheus/tsdb/issues/426
Using `filepath.Join()` instead of strings containing forward slash path delimiters (needed for non-*nix OSes), as suggested by @krasi-georgiev
2018-10-25 10:32:57 +01:00
Chris Marchbanks
2db59a71a6
Fix assert order being backwards in a few places ( #388 )
...
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2018-09-21 09:23:01 +03:00
Fabian Reinartz
22fd3ef24e
Deal with zero-length segments
...
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:34:18 -04:00
Fabian Reinartz
1a5573b4ce
Migrate write ahead log
...
On startup, rewrite the old write ahead log into the new format once.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:34:18 -04:00
Mario Trangoni
c0e888e82b
fix megacheck issues: os.SEEK_SET is deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd.
2018-03-21 22:39:15 +01:00
Krasi Georgiev
eb6cdbcb5b
some test funcitons didn't cleanup after themselves.
2018-02-23 18:45:59 +02:00
Nipun
6537ee832c
Incorporated review comments
2018-01-09 10:09:23 +05:30
Nipun
cadae47522
fixed an issue in TestWALRestoreCorrupted
2018-01-07 08:05:50 +05:30
Fabian Reinartz
adb6be73dd
move label read function int labels pkg
2017-12-21 11:55:58 +01:00
Callum Styan
8326e410d0
decouple tsdb main.go (and tests) from prometheus/pkg
2017-12-18 15:26:16 -08:00
Callum Styan
ca4e817372
move test utils to it's own package so we can use it in tests for other packages in tsdb repo
2017-12-06 17:06:14 -08:00
Callum Styan
6c8bbf3f61
use test utils in wal_test
2017-12-06 16:16:42 -08:00
ranbochen
a27cf34a36
fix bugs on platform windows to pass all test case. ( #192 )
...
* fix bugs on platform windows to pass all test case.
* fix bugs on platform windows to pass all test case
* clean up codes
2017-10-31 15:37:41 +01:00
Fabian Reinartz
80055bb95b
Truncate segments on broken header
2017-10-20 13:16:44 +02:00
Fabian Reinartz
d3682d701c
wal: decode and process in separate threads.
2017-10-06 14:46:52 +02:00
Bas Harenslak
5e1c258a98
Instrument WAL fsync
2017-10-04 22:17:02 +02:00
Fabian Reinartz
1e88ba06b4
Use boolean function instead of postings to drop WAL series
...
There is not guarantee or requirement for WAL writers to only add
series entries in increasing order of IDs. A postings list cannot look
back and thus unordered WAL entries would skip over IDs to not truncate
from the WAL.
We replace it with a simple boolean check function that does not require
order.
2017-09-21 13:31:01 +02:00
Fabian Reinartz
3d8be398d6
Merge branch 'locks' of github.com:prometheus/tsdb into locks
2017-09-11 10:39:55 +02:00
Fabian Reinartz
24362567b9
Fix test flakes
2017-09-11 10:33:17 +02:00
Fabian Reinartz
6892fc6dcb
Finish old WAL segment async, default to no fsync
...
We were still fsyncing while holding the write lock when we cut a new
segment. Given we cannot do anything but logging errors, we might just
as well complete segments asynchronously.
There's not realistic use case where one would fsync after every WAL
entry, thus make the default of a flush interval of 0 to never fsync
which is a much more likely use case.
2017-09-08 18:41:12 +02:00
Fabian Reinartz
c2916736be
Re-use slices on deocding the WAL. Fix tests.
2017-09-08 08:17:01 +02:00
Fabian Reinartz
0db4c227b7
Fix min/max time handling and concurrent crc32 usage
2017-09-07 13:04:02 +02:00
Fabian Reinartz
970bffec8d
Fix WAL errors and add tests for it
2017-09-07 10:58:34 +02:00
Fabian Reinartz
33e9bdf403
WAL refactoring and truncation fixes and test
2017-09-06 14:59:25 +02:00
Fabian Reinartz
1ddedf2b30
Change series ID from uint32 to uint64
2017-09-04 16:08:38 +02:00
Goutham Veeramachaneni
1698c516ad
[WIP]: WAL implementation
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-04 14:52:40 +02:00
Fabian Reinartz
5cf2662074
Refactor WAL into Head and misc improvements
2017-09-01 11:50:58 +02:00
Fabian Reinartz
215a856f34
Just check for general error in WAL test
...
Go 1.9 changed the returned error. Checking the exact message is too
fragile.
2017-08-27 16:57:43 +02:00
Goutham Veeramachaneni
7438ed7035
Expose Intervals type for use by TombstoneReader.
...
TombstoneReader is exposed but Intervals is not.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-08-25 16:06:36 +05:30
Goutham Veeramachaneni
44e9ae38b5
Incorporate PR feedback.
...
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-26 21:26:31 +05:30
Goutham Veeramachaneni
bacb143b7e
Simplify tombstone and WAL Delete formats.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-24 13:42:56 +05:30
Goutham Veeramachaneni
f29fb62fba
Make TombstoneReader a Getter.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-24 11:24:24 +05:30
Goutham Veeramachaneni
3eb4119ab1
Make HeadBlock use WAL.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-23 16:15:16 +05:30
Fabian Reinartz
fb5c5535fc
Misc cleanup
2017-05-18 16:38:45 +02:00
Fabian Reinartz
4862b261d0
Abstract WAL into interface
2017-05-13 17:09:26 +02:00
Fabian Reinartz
535532ca02
Export refdSample
...
The type was part of a exported method signatures and should therefore
be exported as well.
2017-05-12 17:06:26 +02:00
Fabian Reinartz
778103b450
Add liecence file and headers
2017-04-10 20:59:45 +02:00
Fabian Reinartz
10c7c9acbe
Adjust import names to new repository organisation
2017-04-04 11:27:26 +02:00
Fabian Reinartz
ad5812d83a
Handle WAL corruption by truncating
...
This adds handling for various corruption scenarios of the WAL.
If corruption is encountered, we truncate the WAL after the last valid
entry transparently and continue appending after the offset.
2017-03-15 11:16:16 +01:00
Fabian Reinartz
d3edfb5540
Test multiple full write/restore cylcles in WAL
2017-03-08 21:02:08 +01:00
Fabian Reinartz
9c7a88223e
Add full encode/decode WAL cycle test
2017-02-14 21:55:50 -08:00
Fabian Reinartz
2c97428a79
Add basic WAL tests
2017-02-14 16:12:56 -08:00
Fabian Reinartz
9ddbd64d00
Move stats into meta.json file, cleanup, docs
2017-01-19 11:22:47 +01:00
Fabian Reinartz
2f02f86b62
Fix WAL tests
2017-01-19 08:48:11 +01:00
Fabian Reinartz
1943f8d1bb
Fix head block stats races
2017-01-07 18:02:17 +01:00
Fabian Reinartz
937cdb579c
Switch to sequential block names
...
This changes block directory names from the int64 timestamp
to sequential numbering.
2017-01-06 10:45:03 +01:00