Commit graph

11937 commits

Author SHA1 Message Date
Alan Protasio a15e884e7a
Prevent Prometheus from overallocating memory on subquery with large amount of steps. (#12734)
* change initial points slice size

Signed-off-by: Alan Protasio <alanprot@gmail.com>

* refactor on the steps calculation and moving the getXPoint/putXPoint method to the evaluator

Signed-off-by: Alan Protasio <alanprot@gmail.com>

* prevent potential panic

Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update promql/engine.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update promql/engine.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update promql/engine.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update promql/engine.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Allocating slice with maximum size of 5k

Signed-off-by: Alan Protasio <alanprot@gmail.com>

* adding comments

Signed-off-by: Alan Protasio <alanprot@gmail.com>

---------

Signed-off-by: Alan Protasio <alanprot@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2023-09-25 20:15:41 +01:00
Linas Medziunas cbd01fc296 Fix NaN sum check in [Float]Histogram.Equals method
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-09-25 16:03:55 +03:00
ouyang1204 5d233df7ef
Fix rule check broken (#12715)
Signed-off-by: DrAuYueng <ouyang1204@gmail.com>
2023-09-25 17:48:05 +10:00
Goutham Veeramachaneni 86729d4d7b
Update exp package (#12650) 2023-09-21 22:53:51 +02:00
Björn Rabenstein f8dd8770ac
Merge pull request #12757 from bboreham/reuse-bufiter
TSDB: re-use iterator when moving between series
2023-09-21 14:08:53 +02:00
Björn Rabenstein ade2ee3fe4
Merge pull request #12716 from bboreham/simplify-seek
storage: simplify Seek on BufferedSeriesIterator
2023-09-21 13:56:29 +02:00
Dimitar Dimitrov 1155d736b6
Improve sensitivity of TestQuerierIndexQueriesRace
Currently, the two goroutines race against each other and it's possible that the main test goroutine finishes way earlier than appendSeries has had a chance to run at all.

I tested this change by breaking the code that X fixed and running the race test 100 times. Without the additional time.Sleep the test failed 11 times. With the sleep it failed 65 out of the 100 runs. Which is still not ideal, but it's a step forward.

Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
2023-09-21 12:30:08 +02:00
Bryan Boreham 91054875d6
Merge pull request #12732 from bboreham/simplify-rangeeval
promql: simplify inner loop of rangeEval
2023-09-20 20:22:05 +00:00
Bryan Boreham d498bb0d8c
Merge pull request #12875 from dimitarvdimitrov/dimitar/TestQuerierIndexQueriesRace-incorrect-exit-condition
Fix samples generation in TestQuerierIndexQueriesRace

The loop would run forever if the context was cancelled before the goroutine got started.
2023-09-20 20:20:30 +00:00
Dimitar Dimitrov 6f1284ac93 Fix exit condition of TestQuerierIndexQueriesRace
The test was introduced in # but was changed during the code review and not reran with the faulty code since then.

Closes #

Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
2023-09-20 20:22:26 +01:00
Bryan Boreham bca9a79bca
Merge pull request #12877 from bboreham/fix-promtool
promtool: fix compile error from bad merge
2023-09-20 19:14:30 +00:00
Bryan Boreham 5ecea3c840 promtool: fix compile error from bad merge
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-09-20 16:34:20 +00:00
Björn Rabenstein 864da019cd
Merge pull request #12874 from krajorama/outof-order-chunks
Fix duplicate sample detection at chunk size limit
2023-09-20 18:01:21 +02:00
Björn Rabenstein 9071913fd9
Merge pull request #12831 from aknuds1/arve/posting-context
Add context argument to `tsdb.PostingsForMatchers`
2023-09-20 17:15:15 +02:00
Bryan Boreham 2b77ad4591
Merge pull request #12677 from bboreham/retry-on-500
remote-write: respect Retry-After header on 5xx errors
2023-09-20 14:02:34 +00:00
György Krajcsovits 9dbd100a5e Refactor solution to not repeat code
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-09-20 15:54:00 +02:00
György Krajcsovits 96d03b6f46 Fix duplicate sample detection at chunks size limit
Before cutting a new XOR chunk in case the chunk goes over the size
limit, check that the timestamp is in order and not equal or older
than the latest sample in the old chunk.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-09-20 14:49:56 +02:00
György Krajcsovits 56b3a015b6 Add regression test for duplicate detection at chunk size limit
TestHeadDetectsDuplcateSampleAtSizeLimit tests a regression where a
 duplicate sample,is appended to the head, right when the head chunk is
 at the size limit. The test adds all samples as duplicate, thus
 expecting that the result has exactly half of the samples.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-09-20 14:32:20 +02:00
Ben Ye c78124427e
Support specifying series matchers to analyze tsdb (#12842)
* support specifying series matchers to analyze tsdb

Signed-off-by: Ben Ye <benye@amazon.com>

* fix cli docs

Signed-off-by: Ben Ye <benye@amazon.com>

---------

Signed-off-by: Ben Ye <benye@amazon.com>
2023-09-20 11:37:32 +01:00
William Dumont ce6ad15422 remote-write: TestClientRetryAfter status code 500
and compare the retryAfter values.

Signed-off-by: William Dumont <william.dumont@grafana.com>
2023-09-20 10:25:43 +00:00
William Dumont febd62a23e remote-write: refactor TestClientRetryAfter
The new version features a set of test cases that simplify the addition
of new HTTP status codes.

Signed-off-by: William Dumont <william.dumont@grafana.com>
2023-09-20 10:24:52 +00:00
Bryan Boreham 9b85354acd remote-write: respect Retry-After header on 5xx errors
If the server sent it to us, we should assume it knows better than we
do and respect it.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-09-20 10:14:38 +00:00
Paschalis Tsilias c173cd57c9
Add a header to count retried remote write requests (#12729)
Header name is `Retry-Attempt`, only set when >0.

Signed-off-by: Marc Tuduri <marctc@protonmail.com>
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
2023-09-20 11:11:03 +01:00
Björn Rabenstein 83891135c6
Merge pull request #12838 from krajorama/fix-disappearing-span-panic
Fix counterResetInAnyBucket panic
2023-09-19 17:10:27 +02:00
George Krajcsovits 3512b2d678
storage: make histogram reset handling consistent in chainSampleIterator (#12779)
storage: make histogram reset handling consistent in chainSampleIterator

---------

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-09-19 17:06:46 +02:00
Alan Protasio 959c98441b Add context argument to tsdb.PostingsForMatchers
Signed-off-by: Alan Protasio <alanprot@gmail.com>
2023-09-16 18:13:32 +02:00
zenador 69edd8709b
Add warnings (and annotations) to PromQL query results (#12152)
Return annotations (warnings and infos) from PromQL queries

This generalizes the warnings we have already used before (but only for problems with remote read) as "annotations".

Annotations can be warnings or infos (the latter could be false positives). We do not treat them different in the API for now and return them all as "warnings". It would be easy to distinguish them and return infos separately, should that appear useful in the future.

The new annotations are then used to create a lot of warnings or infos during PromQL evaluations. Partially these are things we have wanted for a long time (e.g. inform the user that they have applied `rate` to a metric that doesn't look like a counter), but the new native histograms have created even more needs for those annotations (e.g. if a query tries to aggregate float numbers with histograms).

The annotations added here are not yet complete. A prominent example would be a warning about a range too short for a rate calculation. But such a warnings is more tricky to create with good fidelity and we will tackle it later.

Another TODO is to take annotations into account when evaluating recording rules.

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-09-14 18:57:31 +02:00
Arve Knudsen 156222cc50
Add context argument to LabelQuerier.LabelValues (#12665)
Add context argument to LabelQuerier.LabelValues and
LabelQuerier.SortedLabelValues.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-14 16:02:04 +02:00
Julien Pivotto e21ff116d8
Merge pull request #12841 from pnacht/pinned-gha
Hash-pin workflow GitHub Actions
2023-09-14 10:58:02 +02:00
Arve Knudsen a964349e97
Add context argument to LabelQuerier.LabelNames (#12666)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-14 10:39:51 +02:00
Björn Rabenstein 8f52477e20
Merge pull request #12839 from prometheus/beorn7/histogram
storage: remove obsolete TODO
2023-09-13 18:03:15 +02:00
Arve Knudsen 4451ba10b4
Add context argument to IndexReader.Postings (#12667)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-13 17:45:06 +02:00
Pedro Nacht 35a7ae6eb3
Ensure dependabot updates workflows in ./scripts
Signed-off-by: Pedro Kaj Kjellerup Nacht <pnacht@google.com>
2023-09-13 13:52:37 +00:00
Pedro Nacht 142c829fce
Hash-pin GitHub Actions
Signed-off-by: Pedro Kaj Kjellerup Nacht <pnacht@google.com>
2023-09-13 13:52:14 +00:00
Arve Knudsen 6ef9ed0bc3
Add context argument to DB.Delete (#12834)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-13 15:43:06 +02:00
György Krajcsovits b2fa4d910a Fix more counterResetInAnyBucket edgecases
Case a) empty span is at the beginning of the spans.
Case b) two consequtive empty spans with positive offsets.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-09-13 15:18:40 +02:00
beorn7 0521ec12af storage: remove obsolete TODO
This was solved one layer deeper with #11687.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-09-13 15:10:22 +02:00
Fiona Liao 4419399e4e
Do WBL mmap marker replay concurrently (#12801)
* Benchmark WBL

Extended WAL benchmark test with WBL parts too - added basic cases for
OOO handling - a percentage of series have a percentage of samples set
as OOO ones.

Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com>
2023-09-12 21:31:10 +02:00
Shirley d3a1044354
WBL loading: don't send empty buffers over chan (#12808)
Signed-off-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>
Co-authored-by: Fiona Liao <fiona.y.liao@gmail.com>
2023-09-12 16:26:02 +02:00
Arve Knudsen 6daee89e5f
Add context argument to Querier.Select (#12660)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-12 12:37:38 +02:00
Julius Volz aa7bf083e9
Move /targets page discovered labels to expandable section (#12824)
* Move /targets page discovered labels to expandable section

The current tooltip for showing the pre-relabeling discovered labels for a
target is notoriously unreliable and can get cut off when there are many
labels. This PR introduces a (hopefully unobtuse enough) expander/collapser
button for the discovered labels of each target, and then the discovered labels
are shown in a more persistent way underneath the final target labels, instead
of using a tooltip.

Fixes https://github.com/prometheus/prometheus/issues/9175#issuecomment-1713074341

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Remove obsolete test snapshot

Signed-off-by: Julius Volz <julius.volz@gmail.com>

---------

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2023-09-11 11:36:44 +02:00
Julien Pivotto 7c75d233d0
Merge pull request #12781 from prometheus/dependabot/go_modules/k8s-io-6fdefe01aa
build(deps): bump the k8s-io group with 2 updates
2023-09-08 15:47:41 +02:00
Julien Pivotto d659e672f1
Merge pull request #12816 from slashpai/lightsail
LIGHTSAIL SD: Validate HTTP config
2023-09-08 15:01:35 +02:00
Julien Pivotto 738e0dbaf0
Merge pull request #12814 from slashpai/puppetdb
PUPPETDB SD: Validate HTTP config
2023-09-08 15:00:21 +02:00
Julien Pivotto a2208b2849
Merge pull request #12815 from slashpai/Uyuni
UYUNI SD: Validate HTTP config
2023-09-08 14:59:50 +02:00
Jayapriya Pai 57988fda75
LIGHTSAIL SD: Validate HTTP config
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
2023-09-08 17:54:58 +05:30
Jayapriya Pai 344f9d8d15
PUPPETDB SD: Validate HTTP config
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
2023-09-08 17:42:01 +05:30
Julien Pivotto d27cb0564d
Merge pull request #12802 from bboreham/merge-2-47-0
Merge release 2.47.0 into main
2023-09-08 14:07:05 +02:00
Julien Pivotto 284ba3426b
Merge pull request #12758 from bboreham/trim-rangequery-benchmarks
PromQL: reduce numbers of benchmarks
2023-09-08 14:06:21 +02:00
Jayapriya Pai 34c8dcc73f
UYUNI SD: Validate HTTP config
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
2023-09-08 17:34:41 +05:30