mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-24 21:24:05 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

[BUGFIX] TSDB: Head compaction race condition

Browse source 
There was a small time window where things can go wrong, if a query
starts after `WaitForPendingReadersInTimeRange` returns, but before
`minTime` is stored.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
This commit is contained in:
Bryan Boreham 2024-10-04 16:25:27 -04:00
parent a1c2f59bc5
commit 1b3ac23526
1 changed files with 5 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
tsdb/head.go
View file

@ -1149,14 +1149,16 @@ func (h *Head) truncateMemory(mint int64) (err error) {
h.memTruncationCallBack() h.memTruncationCallBack()
} }
// Set the minTime that new queries will see, before waiting for queries to finish.
h.minTime.Store(mint)
// Don't allow samples to be added before this time.
h.minValidTime.Store(mint)
// We wait for pending queries to end that overlap with this truncation. // We wait for pending queries to end that overlap with this truncation.
if initialized { if initialized {
h.WaitForPendingReadersInTimeRange(h.MinTime(), mint) h.WaitForPendingReadersInTimeRange(h.MinTime(), mint)
} }
h.minTime.Store(mint)
h.minValidTime.Store(mint)
// Ensure that max time is at least as high as min time. // Ensure that max time is at least as high as min time.
for h.MaxTime() < mint { for h.MaxTime() < mint {
h.maxTime.CompareAndSwap(h.MaxTime(), mint) h.maxTime.CompareAndSwap(h.MaxTime(), mint)