Goutham Veeramachaneni 1af81dc5c9 Update sent timestamp when write irrecoverably fails. ... We have an alert that fires when prometheus_remote_storage_highest_timestamp_in_seconds - prometheus_remote_storage_queue_highest_sent_timestamp_seconds becomes too high. But we have an agent that fires this when the remote "rate-limits" the user. This is because prometheus_remote_storage_queue_highest_sent_timestamp_seconds doesn't get updated when the remote sends a 429. I think we should update the metrics, and the change I made makes sense. Because if the requests fails because of connectivity issues, etc. we will never exit the `sendWriteRequestWithBackoff` function. It only exits the function when there is a non-recoverable error, like a bad status code, and in that case, I think the metric needs to be updated. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>		2022-01-03 11:13:48 +01:00
..
chunked.go
chunked_test.go
client.go
client_test.go
codec.go
codec_test.go
ewma.go
intern.go
intern_test.go
max_timestamp.go
metadata_watcher.go
metadata_watcher_test.go
queue_manager.go	Update sent timestamp when write irrecoverably fails.	2022-01-03 11:13:48 +01:00
queue_manager_test.go
read.go
read_handler.go
read_handler_test.go
read_test.go
storage.go
storage_test.go
write.go
write_handler.go
write_handler_test.go
write_test.go