Commit graph

833 commits

Author SHA1 Message Date
Krasi Georgiev d7492b9350
more descriptive var names and some more logging. (#405)
* more descriptive checkpoint var names and some more logging.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-10-11 18:23:52 +03:00
Camille Janicki 0ce41118ed Add msg parameter to Equals function in testutil (#398)
* Add msg parameter to Equals function in testutil

Co-authored-by: Chris Marchbanks <csmarchbanks@gmail.com>
Signed-off-by: Camille Janicki <camille.janicki@gmail.com>
2018-10-03 11:08:31 +03:00
Kautilya Tripathi 3fd6d2f920 more meaningful names for serializedStringTuples and stringTuples (#377)
more meaningful names for serializedStringTuples and stringTuples structs

Signed-off-by: knrt10 <tripathi.kautilya@gmail.com>
Co-authored-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-10-02 21:03:12 +03:00
Simon Pasquier 043e3bb5f9
Merge pull request #396 from codesome/new-metrics
Add new metrics.
2018-10-01 16:00:01 +02:00
Ganesh Vernekar 2638b587f6
Merge remote-tracking branch 'upstream/master' into new-metrics
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-30 20:20:00 +05:30
Ganesh Vernekar 61d0868966 Fix TestCorrectNumTombstones (#399)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-28 13:26:29 +03:00
Ganesh Vernekar 61b000ee0e
Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-28 15:00:51 +05:30
Catalin Patulea b4c7c80227 Reword chunk references, LSB usually means 'bits'. (#364)
https://en.wikipedia.org/wiki/Bit_numbering#Least_significant_bit

Signed-off-by: Catalin Patulea <catalinp@google.com>
2018-09-27 21:38:02 +03:00
Ganesh Vernekar 6e712963e2 Fix updating of NumTombstones in block.Delete(..) (#385)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-27 14:43:22 +03:00
Ganesh Vernekar 632dfb349e
Add new metrics.
1. 'prometheus_tsdb_wal_truncate_fail' for failed WAL truncation.
2. 'prometheus_tsdb_checkpoint_delete_fail' for failed old checkpoint delete.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-25 18:50:57 +05:30
Callum Styan a971f52ac8 clean up after running repair tests (#372)
implement a `CopyDirs` helper to walk all dirs and subdirs and copy all files within to a destination folder.

Refactor `TestRepairBadIndexVersion` to use the `CopyDirs` so it works on a copy db and remove the copy after the test.
This allows running the test more than once and doesn't confuse git.

Signed-off-by: Callum Stytan <callumstyan@gmail.com>
2018-09-21 20:35:33 +03:00
Martin Chodur dbd765a0e3 Refactor Makefile to use Makefile.common (#324)
make use of the Makefile.common

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
2018-09-21 11:08:38 +03:00
Krasi Georgiev d05611c027
removed some unused code and moved mockSeriesSet in querier_test (#394)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-09-21 11:07:35 +03:00
Krasi Georgiev d38516b1c2
remove unused changes variable (#391)
This was added in
55a9b5428a

and later not used after some refactoring in following PRs.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-09-21 09:24:01 +03:00
Chris Marchbanks 2db59a71a6 Fix assert order being backwards in a few places (#388)
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2018-09-21 09:23:01 +03:00
Ganesh Vernekar e779603633 Reuse testutil (#393)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-21 11:47:59 +05:30
Goutham Veeramachaneni 9c8ca47399
Fix filehandling for windows (#392)
* Fix filehandling for windows

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Fix more windows filehandling issues

Windows: Close files before deleting Checkpoints.

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

Windows: Close writers in case of errors so they can be deleted

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

Windows: Close block so that it can be deleted.

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

Windows: Close file to delete it

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

Windows: Close dir so that it can be deleted.

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

Windows: close files so that they can be deleted.

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Review feedback

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2018-09-21 11:01:22 +05:30
Julius Volz 5ae6c60d39 Handle a bunch of unchecked errors (#365)
As discovered by "gosec".

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-09-20 11:33:52 +03:00
Fabian Reinartz dfcb7d0d50
Merge pull request #389 from gouthamve/wal-corruptio
Make sure WAL Repair can handle wrapped errors
2018-09-19 08:47:24 +02:00
Goutham Veeramachaneni c7d0d10da4
Make sure WAL Repair can handle wrapped errors
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2018-09-19 12:12:25 +05:30
Björn Rabenstein d1a6edae6f
Merge pull request #386 from prometheus/beorn7/client_golang
Revert "Remove `prometheus_` prefix from metrics"
2018-09-18 19:27:31 +02:00
beorn7 3bc6c670fa Revert "Remove prometheus_ prefix from metrics"
This reverts commit 98fe30438c.

After some discussion, it was concluded that we want the full
`prometheus_tsdb_...` prefix hardcoded in the library.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-18 19:19:19 +02:00
Chris Marchbanks a8966cb53d Fix race condition between gc and committing (#378)
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2018-09-17 19:58:42 +03:00
Callum Styan 722f0ab920 break MigrateWAL into two functions, detection and migration (#371)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2018-09-17 19:30:56 +03:00
Frederic Branczyk 1b651ea7d4
Merge pull request #383 from prometheus/beorn7/client_golang
Remove `prometheus_` prefix from metrics
2018-09-17 15:16:04 +02:00
beorn7 98fe30438c Remove prometheus_ prefix from metrics
This can now be added by users of the library as needed with the new
https://godoc.org/github.com/prometheus/client_golang/prometheus#WrapRegistererWithPrefix

Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-17 14:54:28 +02:00
Krasi Georgiev ad459ca1f4
re-added TestDeleteUntilCurMax TestDelete_e2e (#382)
no logic changes just uncommented and fixed the issues.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-09-17 14:28:55 +03:00
Bob Shannon cb7f320d42 Expose prometheus_tsdb_lowest_timestamp metric (#363)
* Expose prometheus_tsdb_start_time_seconds metric

Signed-off-by: Bob Shannon <bshannon@palantir.com>

* Search for block with smallest minTime

Signed-off-by: Bob Shannon <bshannon@palantir.com>

* PR comments

Signed-off-by: Bob Shannon <bshannon@palantir.com>

* PR comment: Make metric name more accurate

Signed-off-by: Bob Shannon <bshannon@palantir.com>
2018-09-14 17:37:45 +05:30
Alexey Palazhchenko 9be883670e fix the test data file path in benchmarks. (#345)
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2018-09-13 19:34:26 +03:00
Ganesh Vernekar 2945db18ca Changes in series names (and types) exposed (#376)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-12 14:39:02 +05:30
Goutham Veeramachaneni d9129adb52
Merge pull request #375 from codesome/symbol-table-size
Added metric for symbol table size
2018-09-11 17:43:30 +05:30
Ganesh Vernekar 77d2c7cf3d
Added metric for symbol table size
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-08 23:58:36 +05:30
Simon Pasquier eeb3a7435a
Merge pull request #319 from simonpasquier/improve-repair-err
repair: improve error message
2018-08-07 16:40:27 +02:00
Fabian Reinartz 06f01d45ad
Merge pull request #366 from prometheus/32bit
Fixes for 32bit archs
2018-08-07 06:25:08 -05:00
Fabian Reinartz 22cae653d8 Fixes for 32bit archs
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-08-07 06:52:16 -04:00
Fabian Reinartz 2a0e96eb19
Merge pull request #332 from prometheus/newwal
wal: add write ahead log package
2018-08-07 04:14:15 -05:00
Fabian Reinartz 74b3501161 Fix Rename call
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-08-03 11:25:27 -04:00
Fabian Reinartz 76990518d3
Merge pull request #340 from prometheus/wal_migrate
Migrate write ahead log
2018-08-02 16:58:17 -05:00
Fabian Reinartz ee7ee059ef Fix doc comments
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-08-02 17:57:34 -04:00
Fabian Reinartz a9a8fabd2c
Merge pull request #339 from prometheus/inittime
Properly initialize head time
2018-08-02 16:54:38 -05:00
Fabian Reinartz f8ec0074e7 Add Replace function
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-08-02 17:51:49 -04:00
Fabian Reinartz b81e0fbf2a Address comments
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-20 02:26:12 -04:00
Fabian Reinartz 45071c657c Properly initialize head time
This fixes various issues when initializing the head time range
under different starting conditions.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:41:02 -04:00
Fabian Reinartz 22fd3ef24e Deal with zero-length segments
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:34:18 -04:00
Fabian Reinartz 92e1b20957 Fix close handling
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:34:18 -04:00
Fabian Reinartz 1a5573b4ce Migrate write ahead log
On startup, rewrite the old write ahead log into the new format once.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:34:18 -04:00
Fabian Reinartz 3e76f0163e Address comments
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:25:30 -04:00
Fabian Reinartz 0ad2b8a349 docs: add new WAL format
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:25:30 -04:00
Fabian Reinartz 3f538817f8 move WAL lock
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:25:30 -04:00
Fabian Reinartz d951140ab8 wal: avoid heap allocation in WAL reader
The buffers we allocated were escaping to the heap, resulting in large
memory usage spikes during startup and checkpointing in Prometheus.
This attaches the buffer to the reader object to prevent this.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-07-19 07:25:30 -04:00