implement a `CopyDirs` helper to walk all dirs and subdirs and copy all files within to a destination folder.
Refactor `TestRepairBadIndexVersion` to use the `CopyDirs` so it works on a copy db and remove the copy after the test.
This allows running the test more than once and doesn't confuse git.
Signed-off-by: Callum Stytan <callumstyan@gmail.com>
* Fix filehandling for windows
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
* Fix more windows filehandling issues
Windows: Close files before deleting Checkpoints.
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
Windows: Close writers in case of errors so they can be deleted
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
Windows: Close block so that it can be deleted.
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
Windows: Close file to delete it
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
Windows: Close dir so that it can be deleted.
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
Windows: close files so that they can be deleted.
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
* Review feedback
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
This reverts commit 98fe30438c.
After some discussion, it was concluded that we want the full
`prometheus_tsdb_...` prefix hardcoded in the library.
Signed-off-by: beorn7 <beorn@soundcloud.com>
* Expose prometheus_tsdb_start_time_seconds metric
Signed-off-by: Bob Shannon <bshannon@palantir.com>
* Search for block with smallest minTime
Signed-off-by: Bob Shannon <bshannon@palantir.com>
* PR comments
Signed-off-by: Bob Shannon <bshannon@palantir.com>
* PR comment: Make metric name more accurate
Signed-off-by: Bob Shannon <bshannon@palantir.com>
This fixes various issues when initializing the head time range
under different starting conditions.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
The buffers we allocated were escaping to the heap, resulting in large
memory usage spikes during startup and checkpointing in Prometheus.
This attaches the buffer to the reader object to prevent this.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
We assume in multiple places that the block list held by DB
has blocks sequential by time.
A regression caused us to hold them ordered by ULID, i.e. by creation
time instead.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
Create checkpoints from a sequence of WAL segments while filtering
out obsolete data. The checkpoint format is again a sequence of WAL
segments, which allows us to reuse the serialization format and
implementation.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
Allow to repair the WAL based on the error returned by a reader
during a full scan over all records.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
This adds a new WAL that's agnostic to the actual record contents.
It's much simpler and should be more resilient than the existing one.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
