LiveReaders are instantiated `number of remote write queues * segments`
times, which would cause double registration of the metrics. Instead
this should be orchestrated by the layer above, instantiating the live
reader.
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
In running Prometheus instances, compressing the records was shown to
reduce disk usage by half while incurring a negligible CPU cost.
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
Testing that createBlock creates blocks that can be opened.
and checking the os.RemoveAll for errors will catch errors for un-closed files under windows.
Many missing .Close() calls were added for fixing failing os.RemoveAll
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
Make WAL live tailer return EOF when the there is a half-written record at the end of the file.
Previously, this would cause an infinite loop as we ignored EOFs when filling the buffer. We now differentiate between EOFs that read >0 bytes, and EOFs that didn't.
Add some more unit tests for tailing a corrupt WAL, and unify interfaces Reader and LiveReader for the purposes of testing.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
