Commit graph

355 commits

Author SHA1 Message Date
Marco Pracucci dceedfd215
Merge remote-tracking branch 'remotes/prometheus/main' into sync-prometheus 2024-01-16 11:42:27 +01:00
zenador 72a8f1084b
Restore more efficient version of NewPossibleNonCounterInfo annotation (#13022)
Restore more efficient version of NewPossibleNonCounterInfo annotation

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-01-16 09:54:16 +01:00
Marco Pracucci f955b69b3b
Merge remote-tracking branch 'remotes/prometheus/main' into sync-upstream 2024-01-05 11:38:57 +01:00
Filip Petkovski 0e1ae1d1ca
Add comment
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-25 11:41:07 +01:00
Filip Petkovski 5df3820c7a
Copy last histogram point
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-25 11:20:51 +01:00
Filip Petkovski 1f69dcfa6b
Fix reusing float histograms
In https://github.com/prometheus/prometheus/pull/13276 we started reusing float histogram objects to reduce allocations in PromQL.
That PR introduces a bug where histogram pointers gets copied to the beginning of the histograms slice,
but are still kept in the end of the slice. When a new histogram is read into the last element,
it can overwrite a previous element because the pointer is the same.

This commit fixes the issue by moving outdated points to the end of the slice
so that we don't end up with duplicate pointers in the same buffer. In other words,
the slice gets rotated so that old objects can get reused.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-14 11:53:58 +01:00
Filip Petkovski bb8363dbb3
Add comment on SampleRingIterator
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-13 08:30:02 +01:00
Filip Petkovski e2a9f8ac0f
Reuse float histogram objects
This commit reduces the memory needed to query native histogram objects
by reusing existing HPoint instances.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-11 08:24:58 +01:00
beorn7 0eb0ca42c5 Update “conventional histogram” → “classic histogram”
Signed-off-by: beorn7 <beorn@grafana.com>
2023-11-29 15:22:58 +01:00
György Krajcsovits d42e296516 Merge remote-tracking branch 'upstream/main' into krajo/merge-upstream 2023-11-02 20:45:05 +01:00
Dimitar Dimitrov 9e3df532d8
Export promql.Engine.FindMinMaxTime
This function is useful to analyze promQL queries. We want to use this in Mimir to record the time range which the query touches.

I also chose to remove the `Engine` receiver because it was unnecessary, and it makes it easier to use, but happy to refactor that if you disagree.

The function is untested on its own. If you prefer to have unit tests now that its exported, I can look into adding some.

Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
2023-11-02 13:17:35 +01:00
Dimitar Dimitrov dba5006d7a
Merge branch 'upstream/main' into dimitar/sync-with-upstream-main 2023-11-01 14:06:11 +01:00
Julien Pivotto f568221610
Merge pull request #13057 from prometheus/release-2.48
Merge release-2.48 back into main
2023-10-31 15:24:39 -04:00
Oleksandr Redko 8e5f0387a2
ci(lint): enable nolintlint and remove redundant comments (#12926)
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-10-31 12:35:13 +01:00
Bryan Boreham 49c5e7afe1 PromQL: reduce garbage in range-query evaluation
The temporary variable was allocated on the heap, and it is unnecessary.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-10-29 19:45:06 +00:00
Jeanette Tan 6341ba7374 Merge remote-tracking branch 'upstream/main' into sync-upstream-20231026 2023-10-26 22:18:24 +08:00
zenador 80e977aae6
Remove NewPossibleNonCounterInfo and minimise creating empty annotations (#13012)
* Remove NewPossibleNonCounterInfo until it can be made more efficient, and avoid creating empty annotations as much as possible

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-10-24 17:36:07 +01:00
Jeanette Tan 48e64fdd4d Continue avoiding creating empty annotations as much as possible 2023-10-21 00:37:11 +08:00
Marc Tuduri af7c31ee10
PR feedback
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
2023-10-18 11:53:50 +02:00
Marc Tuduri 8fededf6ad
promql(histograms): Change sample total calculation for histograms
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
2023-10-18 11:51:11 +02:00
Arve Knudsen 35ab75918a Merge remote-tracking branch 'prometheus/main' into arve/upgrade-exp
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-10-06 16:11:40 +02:00
Alan Protasio a15e884e7a
Prevent Prometheus from overallocating memory on subquery with large amount of steps. (#12734)
* change initial points slice size

Signed-off-by: Alan Protasio <alanprot@gmail.com>

* refactor on the steps calculation and moving the getXPoint/putXPoint method to the evaluator

Signed-off-by: Alan Protasio <alanprot@gmail.com>

* prevent potential panic

Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update promql/engine.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update promql/engine.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update promql/engine.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update promql/engine.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Allocating slice with maximum size of 5k

Signed-off-by: Alan Protasio <alanprot@gmail.com>

* adding comments

Signed-off-by: Alan Protasio <alanprot@gmail.com>

---------

Signed-off-by: Alan Protasio <alanprot@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2023-09-25 20:15:41 +01:00
Arve Knudsen 85df1d4013 Merge remote-tracking branch 'prometheus/main' into chore/sync-prometheus
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-21 09:19:23 +02:00
Bryan Boreham 91054875d6
Merge pull request #12732 from bboreham/simplify-rangeeval
promql: simplify inner loop of rangeEval
2023-09-20 20:22:05 +00:00
Arve Knudsen e48d4e5835 Merge remote-tracking branch 'prometheus/main' into chore/sync-prometheus
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-18 09:29:42 +02:00
zenador 69edd8709b
Add warnings (and annotations) to PromQL query results (#12152)
Return annotations (warnings and infos) from PromQL queries

This generalizes the warnings we have already used before (but only for problems with remote read) as "annotations".

Annotations can be warnings or infos (the latter could be false positives). We do not treat them different in the API for now and return them all as "warnings". It would be easy to distinguish them and return infos separately, should that appear useful in the future.

The new annotations are then used to create a lot of warnings or infos during PromQL evaluations. Partially these are things we have wanted for a long time (e.g. inform the user that they have applied `rate` to a metric that doesn't look like a counter), but the new native histograms have created even more needs for those annotations (e.g. if a query tries to aggregate float numbers with histograms).

The annotations added here are not yet complete. A prominent example would be a warning about a range too short for a rate calculation. But such a warnings is more tricky to create with good fidelity and we will tackle it later.

Another TODO is to take annotations into account when evaluating recording rules.

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-09-14 18:57:31 +02:00
Arve Knudsen 6daee89e5f
Add context argument to Querier.Select (#12660)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-12 12:37:38 +02:00
Dimitar Dimitrov 77ac7ad40a
Merge remote-tracking branch 'upstream/main' into dimitar/pull-upstream 2023-09-05 16:19:00 +02:00
Bryan Boreham e4dd3469ac lint
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-08-28 10:39:16 +01:00
Bryan Boreham 5ce990cabc promql: simplify rangeEval a bit more
We can't have both a float and a histogram at the same timestep.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-08-28 10:28:09 +01:00
Bryan Boreham 3879488476 promql: simplify inner loop of rangeEval
Took out the loops with break after one iteration, and extract some
common code to a function.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-08-21 19:52:14 +01:00
Bryan Boreham 5cea37c069
Merge pull request #12682 from bboreham/contains-same-label-set
promql engine: check unique labels using existing map

ContainsSameLabelset constructs a map with the same hash key as the one used to compile the output of rangeEval, so we can use that one and save work.

Need to hold the timestamp so we can be sure we saw the same series in the same evaluation.
2023-08-14 14:12:47 +01:00
Bryan Boreham 0670e4771a promql engine: check unique labels using existing map
`ContainsSameLabelset` constructs a map with the same hash key as
the one used to compile the output of `rangeEval`, so we can use that
one and save work.

Need to hold the timestamp so we can be sure we saw the same series
in the same evaluation.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-08-13 18:09:10 +01:00
Bryan Boreham 8d47b3d497
Merge pull request #12579 from charleskorn/timestamp
Don't recreate iterator for each series on each timestep when evaluating a query with `timestamp()`
2023-08-05 10:51:38 +01:00
Oleg Zaytsev a54cb4744d Merge branch 'main' into prometheus-2023-07-31-76dd9b547 2023-08-04 09:46:03 +02:00
Charles Korn 6087c555ed
Address PR feedback: clarify comment
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-08-01 13:30:10 +10:00
Charles Korn fb3935e8f9
Address PR feedback: rename method
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-08-01 13:30:07 +10:00
Julius Volz 531567d46e Drop metric name for "atan2" binary operator
The operator changes the meaning of the metric, so the metric name should
be dropped. Technically this would be a breaking change, but it's also very
obviously a bug and not likely that anyone depends on it.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2023-07-24 14:36:02 +02:00
Charles Korn fde6ebb17d
Create per-series iterators only once per selector, rather than recreating it for each time step.
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-07-20 11:24:21 +10:00
Charles Korn 993618adea
Don't create a new iterator for every time step.
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-07-20 11:24:21 +10:00
Charles Korn b114c0888d
Simplify loop
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-07-20 11:24:20 +10:00
Charles Korn a142998052
Expand series set just once
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-07-20 11:24:19 +10:00
Charles Korn eeface2e17
Inline method
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-07-20 11:24:19 +10:00
Charles Korn a2a2cc757e
Extract timestamp special case to its own method.
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-07-20 11:24:18 +10:00
Marco Pracucci 7ad111b27e
Merge remote-tracking branch 'remotes/prometheus/main' into update-upstream 2023-07-06 15:13:54 +02:00
Bryan Boreham 7defd025cb Placate lint
I think the version using scoping was better, but I'm out of energy to fight the linter.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-04 15:01:01 +00:00
Bryan Boreham cfbbd2ce2a promql: include parsing in active-query tracking
So that the max-concurrency limit is applied.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-04 15:01:01 +00:00
Bryan Boreham 1706264cef promql: refactor: create query object before parsing
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-04 15:01:01 +00:00
Bryan Boreham 621d29795d promql: refactor: extract fn to wait on concurrency limit
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-04 15:01:01 +00:00
Giedrius Statkevičius 3f230fc9f8 promql: convert QueryOpts to interface
Convert QueryOpts to an interface so that downstream projects like
https://github.com/thanos-community/promql-engine could extend the query
options with engine specific options that are not in the original
engine.

Will be used to enable query analysis per-query.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
2023-07-03 16:20:31 +03:00