Marek Slabicki
8224ddec23
Capitalizing first letter of all log lines ( #7043 )
...
Signed-off-by: Marek Slabicki <thaniri@gmail.com>
2020-04-11 09:22:18 +01:00
Brad Walker
3348930df5
Replace fileutil.ReadDir with ioutil.ReadDir ( #7029 ) ( #7033 )
...
* tsdb: Replace fileutil.ReadDir with ioutil.ReadDir (#7029 )
Signed-off-by: Brad Walker <brad@bradmwalker.com>
* tsdb: Remove fileutil.ReadDir (#7029 )
Signed-off-by: Brad Walker <brad@bradmwalker.com>
2020-04-06 19:04:20 +05:30
Ben Kochie
269e7c8091
Fix golint issues.
...
Signed-off-by: Ben Kochie <superq@gmail.com>
2020-03-23 20:38:43 +01:00
Ganesh Vernekar
e64a149984
Close Head in DBReadOnly.FlushWAL ( #7022 )
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-03-23 14:49:44 +05:30
zhulongcheng
e813f60fd6
tsdb: fix sequence check for WAL segments ( #7032 )
...
Signed-off-by: zhulongcheng <zhulongcheng.dev@gmail.com>
2020-03-23 13:16:28 +05:30
Julien Pivotto
f1984bb007
Merge pull request #7025 from prometheus/release-2.17
...
Merge release 2.17 into master
2020-03-21 21:39:07 +01:00
Callum Styan
f802f1e8ca
Fix bug with WAL watcher and Live Reader metrics usage. ( #6998 )
...
* Fix bug with WAL watcher and Live Reader metrics usage.
Calling NewXMetrics when creating a Watcher or LiveReader results in a
registration error, which we're ignoring, and as a result other than the
first Watcher/Reader created, we had no metrics for either. So we would
only have metrics like Watcher Records Read for the first remote write
config in a users config file.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2020-03-20 17:34:15 +01:00
zhulongcheng
5f5c7a4477
tsdb: sort checkpoints by segment number ( #6987 )
...
Signed-off-by: zhulongcheng <zhulongcheng.dev@gmail.com>
2020-03-18 20:40:41 +05:30
李国忠
261cbab8e9
remove Unused parameter 'reg' in wal.Open function ( #6941 )
...
Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>
2020-03-10 11:01:47 +05:30
Julien Pivotto
52630ad0c7
Make head Postings only return series in time range
...
benchmark old ns/op new ns/op delta
BenchmarkQuerierSelect/Head/1of1000000-8 405805161 120436132 -70.32%
BenchmarkQuerierSelect/Head/10of1000000-8 403079620 120624292 -70.07%
BenchmarkQuerierSelect/Head/100of1000000-8 404678647 120923522 -70.12%
BenchmarkQuerierSelect/Head/1000of1000000-8 403145813 118636563 -70.57%
BenchmarkQuerierSelect/Head/10000of1000000-8 405020046 125716206 -68.96%
BenchmarkQuerierSelect/Head/100000of1000000-8 426305002 175808499 -58.76%
BenchmarkQuerierSelect/Head/1000000of1000000-8 619002108 567013003 -8.40%
BenchmarkQuerierSelect/SortedHead/1of1000000-8 1276316086 120281094 -90.58%
BenchmarkQuerierSelect/SortedHead/10of1000000-8 1282631170 121836526 -90.50%
BenchmarkQuerierSelect/SortedHead/100of1000000-8 1325824787 121174967 -90.86%
BenchmarkQuerierSelect/SortedHead/1000of1000000-8 1271386268 121025117 -90.48%
BenchmarkQuerierSelect/SortedHead/10000of1000000-8 1280223345 130838948 -89.78%
BenchmarkQuerierSelect/SortedHead/100000of1000000-8 1271401620 243635515 -80.84%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-8 1360256090 1307744674 -3.86%
BenchmarkQuerierSelect/Block/1of1000000-8 748183120 707888498 -5.39%
BenchmarkQuerierSelect/Block/10of1000000-8 741084129 716317249 -3.34%
BenchmarkQuerierSelect/Block/100of1000000-8 722157273 735624256 +1.86%
BenchmarkQuerierSelect/Block/1000of1000000-8 727587744 731981838 +0.60%
BenchmarkQuerierSelect/Block/10000of1000000-8 727518578 726860308 -0.09%
BenchmarkQuerierSelect/Block/100000of1000000-8 765577046 757382386 -1.07%
BenchmarkQuerierSelect/Block/1000000of1000000-8 1126722881 1084779083 -3.72%
benchmark old allocs new allocs delta
BenchmarkQuerierSelect/Head/1of1000000-8 4000018 24 -100.00%
BenchmarkQuerierSelect/Head/10of1000000-8 4000036 82 -100.00%
BenchmarkQuerierSelect/Head/100of1000000-8 4000216 625 -99.98%
BenchmarkQuerierSelect/Head/1000of1000000-8 4002016 6028 -99.85%
BenchmarkQuerierSelect/Head/10000of1000000-8 4020016 60037 -98.51%
BenchmarkQuerierSelect/Head/100000of1000000-8 4200016 600047 -85.71%
BenchmarkQuerierSelect/Head/1000000of1000000-8 6000016 6000016 +0.00%
BenchmarkQuerierSelect/SortedHead/1of1000000-8 4000055 28 -100.00%
BenchmarkQuerierSelect/SortedHead/10of1000000-8 4000073 87 -100.00%
BenchmarkQuerierSelect/SortedHead/100of1000000-8 4000253 630 -99.98%
BenchmarkQuerierSelect/SortedHead/1000of1000000-8 4002053 6036 -99.85%
BenchmarkQuerierSelect/SortedHead/10000of1000000-8 4020053 60054 -98.51%
BenchmarkQuerierSelect/SortedHead/100000of1000000-8 4200053 600074 -85.71%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-8 6000053 6000053 +0.00%
BenchmarkQuerierSelect/Block/1of1000000-8 6000021 6000021 +0.00%
BenchmarkQuerierSelect/Block/10of1000000-8 6000057 6000057 +0.00%
BenchmarkQuerierSelect/Block/100of1000000-8 6000417 6000417 +0.00%
BenchmarkQuerierSelect/Block/1000of1000000-8 6004017 6004017 +0.00%
BenchmarkQuerierSelect/Block/10000of1000000-8 6040017 6040017 +0.00%
BenchmarkQuerierSelect/Block/100000of1000000-8 6400017 6400017 +0.00%
BenchmarkQuerierSelect/Block/1000000of1000000-8 10000018 10000018 +0.00%
benchmark old bytes new bytes delta
BenchmarkQuerierSelect/Head/1of1000000-8 176001177 1392 -100.00%
BenchmarkQuerierSelect/Head/10of1000000-8 176002329 4368 -100.00%
BenchmarkQuerierSelect/Head/100of1000000-8 176013849 33520 -99.98%
BenchmarkQuerierSelect/Head/1000of1000000-8 176129056 321456 -99.82%
BenchmarkQuerierSelect/Head/10000of1000000-8 177281049 3427376 -98.07%
BenchmarkQuerierSelect/Head/100000of1000000-8 188801049 35055408 -81.43%
BenchmarkQuerierSelect/Head/1000000of1000000-8 304001059 304001049 -0.00%
BenchmarkQuerierSelect/SortedHead/1of1000000-8 229192188 2488 -100.00%
BenchmarkQuerierSelect/SortedHead/10of1000000-8 229193340 5568 -100.00%
BenchmarkQuerierSelect/SortedHead/100of1000000-8 229204860 35536 -99.98%
BenchmarkQuerierSelect/SortedHead/1000of1000000-8 229320060 345104 -99.85%
BenchmarkQuerierSelect/SortedHead/10000of1000000-8 230472060 3894672 -98.31%
BenchmarkQuerierSelect/SortedHead/100000of1000000-8 241992060 40511632 -83.26%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-8 357192060 357192060 +0.00%
BenchmarkQuerierSelect/Block/1of1000000-8 227201516 227201506 -0.00%
BenchmarkQuerierSelect/Block/10of1000000-8 227203057 227203041 -0.00%
BenchmarkQuerierSelect/Block/100of1000000-8 227217161 227217165 +0.00%
BenchmarkQuerierSelect/Block/1000of1000000-8 227358279 227358289 +0.00%
BenchmarkQuerierSelect/Block/10000of1000000-8 228769485 228769475 -0.00%
BenchmarkQuerierSelect/Block/100000of1000000-8 242881487 242881477 -0.00%
BenchmarkQuerierSelect/Block/1000000of1000000-8 384001705 384001705 +0.00%
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-02-20 22:41:46 +01:00
johncming
c30abf1e2b
tsdb/wal: remove unused argument.
...
Signed-off-by: johncming <johncming@yahoo.com>
2020-02-15 21:54:09 +08:00
John McBride
669592a2c4
Exports metric for WAL write errors ( #6647 )
...
* Exports metric for WAL write errors
Signed-off-by: John McBride <jpmmcbride@gmail.com>
* Correct name for counter
Signed-off-by: John McBride <jpmmcbride@gmail.com>
* Move WAL write failure to wal.go
Signed-off-by: John McBride <jpmmcbride@gmail.com>
* WAL write fail metric moved to Log for external consumers
Signed-off-by: John McBride <jpmmcbride@gmail.com>
2020-01-17 12:12:04 -08:00
Julien Pivotto
201491fd1b
Remove old checkpoint dir if it still exists ( #6621 )
...
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-14 09:35:24 +00:00
Brian Brazil
536d416299
Fix tsdb code and tests to work on Windows. ( #6547 )
...
Add back Windows CI, we lost it when tsdb was merged into the prometheus
repo. There's many tests failing outside tsdb, so only test tsdb for
now.
Fixes #6513
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-01-04 14:55:02 +00:00
Josh Soref
91d76c8023
Spelling ( #6517 )
...
* spelling: alertmanager
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: attributes
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: autocomplete
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: bootstrap
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: caught
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: chunkenc
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: compaction
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: corrupted
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: deletable
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: expected
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: fine-grained
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: initialized
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: iteration
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: javascript
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: multiple
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: number
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: overlapping
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: possible
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: postings
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: procedure
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: programmatic
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: queuing
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: querier
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: repairing
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: received
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: reproducible
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: retention
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: sample
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: segements
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: semantic
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: software [LICENSE]
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: staging
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: timestamp
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: unfortunately
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: uvarint
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: subsequently
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: ressamples
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-01-02 15:54:09 +01:00
Callum Styan
6a24eee340
Simplify duration check for watcher WAL replay.
...
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-11-26 16:53:11 -08:00
Callum Styan
2d3ce3916c
Log the start and end of the WAL replay within the WAL watcher.
...
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-11-19 14:21:13 -08:00
Tom Wilkie
de0a772b8e
Port tsdb to use pkg/labels. ( #6326 )
...
* Port tsdb to use pkg/labels.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Get tests passing.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Remove useless cast.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Appease linters.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-11-18 11:53:33 -08:00
Dipack P Panjabi
ce7bab04dd
Compute WAL size and use it during retention size checks ( #5886 )
...
* Compute WAL size and account for it when applying the retention settings.
Signed-off-by: Dipack P Panjabi <dpanjabi@hudson-trading.com>
2019-11-12 09:40:16 +07:00
Chris Marchbanks
8df4bca470
Garbage collect asynchronously in the WAL Watcher
...
The WAL Watcher replays a checkpoint after it is created in order to
garbage collect series that no longer exist in the WAL. Currently the
garbage collection process is done serially with reading from the tip of
the WAL which can cause large delays in writing samples to remote
storage just after compaction occurs.
This also fixes a memory leak where dropped series are not cleaned up as
part of the SeriesReset process.
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2019-10-07 14:36:10 -06:00
陈谭军
103f26d188
fix the wrong word ( #6069 )
...
Signed-off-by: chentanjun <2799194073@qq.com>
2019-09-30 09:54:55 -06:00
AllenZMC
1e62435960
fix wrong spells in live_reader.go ( #5899 )
...
* fix wrong spells in live_reader.go
* fix wrong spells in lex.go
Signed-off-by: czm <zhongming.chang@daocloud.io>
2019-09-21 16:36:33 +03:00
johncming
612f9cb361
tsdb/wal: pull out wal metrics separately as tsdb.DB ( #5957 )
...
Signed-off-by: johncming <johncming@yahoo.com>
2019-09-19 14:24:34 +03:00
Callum Styan
3344bb5c33
Move WAL watcher code to tsdb/wal package. ( #5999 )
...
* Move WAL watcher code to tsdb/wal package.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* Fix tests after moving WAL watcher code.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* Lint fixes.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-09-19 14:45:41 +05:30
Erfan Besharat
9336c01dfd
Add methods to fetch page's buf data in tsdb WAL ( #5967 )
...
* move the WAL page buf reset in its own func
Signed-off-by: Erfan Besharat <erbesharat@gmail.com>
2019-08-30 11:38:36 +03:00
陈谭军
50d453b3c3
fix-up tsdb-typo ( #5954 )
...
Signed-off-by: chentanjun <2799194073@qq.com>
2019-08-28 14:43:02 +01:00
johncming
7d43feb03f
tsdb/wal: some small refactoring for easier reading ( #5930 )
...
Signed-off-by: johncming <johncming@yahoo.com>
2019-08-22 16:12:59 +03:00
Bartek Plotka
f0863a604e
Removed extra tsdb/testutil after merge.
...
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2019-08-14 10:12:32 +01:00
Ganesh Vernekar
5ecef3542d
Cleanup after merging tsdb into prometheus
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-08-13 14:04:14 +05:30
Ganesh Vernekar
7cf09b0395
Moving tsdb into its own subdirectory
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-08-13 13:58:49 +05:30