* Backfilling: optimize for non-consecutive blocks
When you have missing data for > 2 hours, you spend a lot of time
re-reading the complete file. It is not optimal.
This introduces a fastpath for this scenario.
Next, we do parse the metric even when we know we will not use it, based
on its timestamp. This only computes the metric when we know its
timestamp is right.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
I initially thought I could somehow rescue the current column layout
by recycling the tabwriter, but flushing completely blanks
it. However, by setting a minimum width of 13, we get a slightly
broader DURATION column but otherwise nice formatting, unless numbers
get really big, but that's OK, I guess.
Before:
```
BLOCK ULID MIN TIME MAX TIME DURATION NUM SAMPLES NUM CHUNKS NUM SERIES SIZE
01ETN0KGNP5WWK9T5QMQGBG9F1 2020-11-19 07:39:17 +0000 UTC 2020-11-19 07:44:17 +0000 UTC 5m0.001s 8 2 2 624B
01ETN0KGQSFF0AB2QDZVQG3CWC 2020-11-19 10:25:57 +0000 UTC 2020-11-19 10:30:57 +0000 UTC 5m0.001s 8 2 2 622B
01ETN0KGSW8KYP3YPG4X20P60Z 2020-11-19 13:12:37 +0000 UTC 2020-11-19 13:17:37 +0000 UTC 5m0.001s 8 2 2 625B
```
After:
```
BLOCK ULID MIN TIME MAX TIME DURATION NUM SAMPLES NUM CHUNKS NUM SERIES SIZE
01ETN0R72SXN9A1FG732P7KFFN 2020-11-19 07:39:17 +0000 UTC 2020-11-19 07:44:17 +0000 UTC 5m0.001s 8 2 2 624B
01ETN0R74Y9AG1A1MKN4MZK7WM 2020-11-19 10:25:57 +0000 UTC 2020-11-19 10:30:57 +0000 UTC 5m0.001s 8 2 2 622B
01ETN0R76KXZ5VQECMDNES49J6 2020-11-19 13:12:37 +0000 UTC 2020-11-19 13:17:37 +0000 UTC 5m0.001s 8 2 2 625B
```
After without the `-r` flag:
```
BLOCK ULID MIN TIME MAX TIME DURATION NUM SAMPLES NUM CHUNKS NUM SERIES SIZE
01ETN0RFFJ42274NWR1GH0RTV6 1605771557000 1605771857001 5m0.001s 8 2 2 624
01ETN0RFJ1MZCHHS2SBZS8XC27 1605781557000 1605781857001 5m0.001s 8 2 2 622
01ETN0RFM98N3V4KD2DZXFGHGN 1605791557000 1605791857001 5m0.001s 8 2 2 625
```
Signed-off-by: beorn7 <beorn@grafana.com>
* Small cleanup of API code.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed typo spotted by Julien.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
This also caches credentials that are obtained e.g. via IRSA on AWS EKS.
Previously, every refresh cycle would request the credentials again.
Signed-off-by: Alfred Krohmer <alfred.krohmer@logmein.com>
Label selector can be
"set-based"(https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#set-based-requirement)
but such a selector causes Prometheus start failure with the "unexpected
error: parsing YAML file ...: invalid selector: 'foo in (bar,baz)';
can't understand 'baz)'"-like error.
This is caused by the `fields.ParseSelector(string)` function that
simply splits an expression as a CSV-list, so a comma confuses such a
parsing method and lead to the error.
Use `labels.Parse(string)` to use a valid lexer to parse a selector
expression.
Closes#8284.
Signed-off-by: Alexey Shumkin <Alex.Crezoff@gmail.com>
* Logging added for when compaction takes more than the block time range
Signed-off-by: arthursens <arthursens2005@gmail.com>
* Log only if no errors were already logged
Signed-off-by: arthursens <arthursens2005@gmail.com>
* Log duration as human readable string
Signed-off-by: arthursens <arthursens2005@gmail.com>
* Move logging from compactHead() to Compact()
Signed-off-by: arthursens <arthursens2005@gmail.com>
* Compute duration of all head compactions plus wal truncation
Signed-off-by: arthursens <arthursens2005@gmail.com>
* Remove named return added os first commits
Signed-off-by: arthursens <arthursens2005@gmail.com>
* Address nits
Signed-off-by: arthursens <arthursens2005@gmail.com>
* Change miliseconds to seconds to make fuzzit tests happy
Signed-off-by: ArthurSens <arthursens2005@gmail.com>
