diff --git a/docs/storage.md b/docs/storage.md index 2142c970f..e625e9c22 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -115,13 +115,12 @@ time series you scrape (fewer targets or fewer series per target), or you can increase the scrape interval. However, reducing the number of series is likely more effective, due to compression of samples within a series. -If your local storage becomes corrupted for whatever reason, the best -strategy to address the problem is to shut down Prometheus then remove the -entire storage directory. You can also try removing individual block directories, -or the WAL directory to resolve the problem. Note that this means losing -approximately two hours data per block directory. Again, Prometheus's local -storage is not intended to be durable long-term storage; external solutions -offer extended retention and data durability. +If your local storage becomes corrupted to the point where Prometheus will not +start it is recommended to backup the storage directory and restore the +corrupted block directories from your backups. If you do not have backups the +last resort is to remove the corrupted files. For example you can try removing +individual block directories or the write-ahead-log (wal) files. Note that this +means losing the data for the time range those blocks or wal covers. CAUTION: Non-POSIX compliant filesystems are not supported for Prometheus' local storage as unrecoverable corruptions may happen. NFS filesystems