mirror of
https://github.com/prometheus/prometheus.git
synced 2024-12-25 05:34:05 -08:00
1af81dc5c9
We have an alert that fires when prometheus_remote_storage_highest_timestamp_in_seconds - prometheus_remote_storage_queue_highest_sent_timestamp_seconds becomes too high. But we have an agent that fires this when the remote "rate-limits" the user. This is because prometheus_remote_storage_queue_highest_sent_timestamp_seconds doesn't get updated when the remote sends a 429. I think we should update the metrics, and the change I made makes sense. Because if the requests fails because of connectivity issues, etc. we will never exit the `sendWriteRequestWithBackoff` function. It only exits the function when there is a non-recoverable error, like a bad status code, and in that case, I think the metric needs to be updated. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> |
||
|---|---|---|
| .. | ||
| chunked.go | ||
| chunked_test.go | ||
| client.go | ||
| client_test.go | ||
| codec.go | ||
| codec_test.go | ||
| ewma.go | ||
| intern.go | ||
| intern_test.go | ||
| max_timestamp.go | ||
| metadata_watcher.go | ||
| metadata_watcher_test.go | ||
| queue_manager.go | ||
| queue_manager_test.go | ||
| read.go | ||
| read_handler.go | ||
| read_handler_test.go | ||
| read_test.go | ||
| storage.go | ||
| storage_test.go | ||
| write.go | ||
| write_handler.go | ||
| write_handler_test.go | ||
| write_test.go | ||