Brian Brazil
0482d93fe6
Add contexts to index writer to fix test races.
...
With recent speed improvements to populate block,
the cancellation test now fails regularly on CI.
Use contexts to get the index writer to shut down
much faster, and that allows us to make the cancellation
test faster too.
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-12-16 17:28:29 +00:00
Brian Brazil
971dafdfbe
Coalesce series reads where we can.
...
When compacting rather than doing a read of all
series in the index per label name, do many at once
but only when it won't use (much) more ram than writing the
special all index does.
original in-memory postings:
BenchmarkCompactionFromHead/labelnames=1,labelvalues=100000-4 1 1202383447 ns/op 158936496 B/op 1031511 allocs/op
BenchmarkCompactionFromHead/labelnames=10,labelvalues=10000-4 1 1141792706 ns/op 154453408 B/op 1093453 allocs/op
BenchmarkCompactionFromHead/labelnames=100,labelvalues=1000-4 1 1169288829 ns/op 161072336 B/op 1110021 allocs/op
BenchmarkCompactionFromHead/labelnames=1000,labelvalues=100-4 1 1115700103 ns/op 149480472 B/op 1129180 allocs/op
BenchmarkCompactionFromHead/labelnames=10000,labelvalues=10-4 1 1283813141 ns/op 162937800 B/op 1202771 allocs/op
before:
BenchmarkCompactionFromHead/labelnames=1,labelvalues=100000-4 1 1145195941 ns/op 131749984 B/op 834400 allocs/op
BenchmarkCompactionFromHead/labelnames=10,labelvalues=10000-4 1 1233526345 ns/op 127889416 B/op 897033 allocs/op
BenchmarkCompactionFromHead/labelnames=100,labelvalues=1000-4 1 1821942296 ns/op 131665648 B/op 914836 allocs/op
BenchmarkCompactionFromHead/labelnames=1000,labelvalues=100-4 1 8035568665 ns/op 123811832 B/op 934312 allocs/op
BenchmarkCompactionFromHead/labelnames=10000,labelvalues=10-4 1 71325926267 ns/op 140722648 B/op 1016824 allocs/op
after:
BenchmarkCompactionFromHead/labelnames=1,labelvalues=100000-4 1 1101429174 ns/op 129063496 B/op 832571 allocs/op
BenchmarkCompactionFromHead/labelnames=10,labelvalues=10000-4 1 1074466374 ns/op 124154888 B/op 894875 allocs/op
BenchmarkCompactionFromHead/labelnames=100,labelvalues=1000-4 1 1166510282 ns/op 128790648 B/op 912931 allocs/op
BenchmarkCompactionFromHead/labelnames=1000,labelvalues=100-4 1 1075013071 ns/op 120570696 B/op 933511 allocs/op
BenchmarkCompactionFromHead/labelnames=10000,labelvalues=10-4 1 1231673790 ns/op 138754288 B/op 1022791 allocs/op
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-12-12 08:38:14 +00:00
Brian Brazil
4df814f509
Don't buffer up tables in memory on compaction. ( #6422 )
...
We can instead write it as we go, and then go back and write in the
length at the end.
Also fix the compaction benchmark, which indicates no changes.
For the benchmark, this brings maximum memory usage of the buffers
from ~200kB down to 128B.
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-12-11 12:49:13 +00:00
Tom Wilkie
de0a772b8e
Port tsdb to use pkg/labels. ( #6326 )
...
* Port tsdb to use pkg/labels.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Get tests passing.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Remove useless cast.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Appease linters.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-11-18 11:53:33 -08:00
yuxiaobo
7850f1b35c
new world spelling mistake
...
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
2019-10-17 19:09:54 +08:00
陈谭军
103f26d188
fix the wrong word ( #6069 )
...
Signed-off-by: chentanjun <2799194073@qq.com>
2019-09-30 09:54:55 -06:00
Callum Styan
3344bb5c33
Move WAL watcher code to tsdb/wal package. ( #5999 )
...
* Move WAL watcher code to tsdb/wal package.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* Fix tests after moving WAL watcher code.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* Lint fixes.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-09-19 14:45:41 +05:30
Bartek Plotka
f0863a604e
Removed extra tsdb/testutil after merge.
...
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2019-08-14 10:12:32 +01:00
Ganesh Vernekar
5ecef3542d
Cleanup after merging tsdb into prometheus
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-08-13 14:04:14 +05:30
Ganesh Vernekar
7cf09b0395
Moving tsdb into its own subdirectory
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-08-13 13:58:49 +05:30
