mirror of
https://github.com/prometheus/prometheus.git
synced 2025-03-05 20:59:13 -08:00
1af81dc5c9
We have an alert that fires when prometheus_remote_storage_highest_timestamp_in_seconds - prometheus_remote_storage_queue_highest_sent_timestamp_seconds becomes too high. But we have an agent that fires this when the remote "rate-limits" the user. This is because prometheus_remote_storage_queue_highest_sent_timestamp_seconds doesn't get updated when the remote sends a 429. I think we should update the metrics, and the change I made makes sense. Because if the requests fails because of connectivity issues, etc. we will never exit the `sendWriteRequestWithBackoff` function. It only exits the function when there is a non-recoverable error, like a bad status code, and in that case, I think the metric needs to be updated. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> |
||
|---|---|---|
| .. | ||
| remote | ||
| buffer.go | ||
| buffer_test.go | ||
| fanout.go | ||
| fanout_test.go | ||
| generic.go | ||
| interface.go | ||
| lazy.go | ||
| memoized_iterator.go | ||
| memoized_iterator_test.go | ||
| merge.go | ||
| merge_test.go | ||
| noop.go | ||
| secondary.go | ||
| series.go | ||
| series_test.go | ||