LiveReaders are instantiated `number of remote write queues * segments`
times, which would cause double registration of the metrics. Instead
this should be orchestrated by the layer above, instantiating the live
reader.
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
In running Prometheus instances, compressing the records was shown to
reduce disk usage by half while incurring a negligible CPU cost.
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
Make WAL live tailer return EOF when the there is a half-written record at the end of the file.
Previously, this would cause an infinite loop as we ignored EOFs when filling the buffer. We now differentiate between EOFs that read >0 bytes, and EOFs that didn't.
Add some more unit tests for tailing a corrupt WAL, and unify interfaces Reader and LiveReader for the purposes of testing.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
