472 commits

Author	SHA1	Message	Date
Björn Rabenstein	1daf7cdd62	Merge pull request #14626 from cuiweiyuan/main ... chore: fix some function names	2024-08-15 11:46:21 +02:00
cuiweiyuan	1800af54f0	chore: fix some function names ... Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>	2024-08-15 13:57:21 +08:00
Charles Korn	52818a97e2	Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms ... # Conflicts: # promql/promqltest/testdata/native_histograms.test	2024-08-14 07:52:08 +10:00
Björn Rabenstein	c2bc6cfe97	Merge pull request #14621 from charleskorn/panic-message ... promql: clarify error message logged when panic occurs during query evaluation	2024-08-13 23:02:43 +02:00
Arve Knudsen	0503d4f372	PromQL: Fix comment regarding non-nil histogram pointer ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-13 08:55:24 +02:00
Charles Korn	f992f81bd0	Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms ... Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com>	2024-08-09 13:58:54 +10:00
Charles Korn	82bb35fabb	Address PR feedback: fix typo and rename variable ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-09 13:51:31 +10:00
Charles Korn	f91009aa2e	promql: clarify error message when panic occurs during query evaluation ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-08 09:11:38 +10:00
Björn Rabenstein	27579c9148	Merge pull request #14605 from krajorama/fix-staleness-pool-corrupt ... Fix histogram pool poisoning bug chunkenc.Iterator	2024-08-07 21:02:08 +02:00
George Krajcsovits	17b0b788da	Update promql/engine.go ... Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>	2024-08-07 20:15:46 +02:00
Charles Korn	0f4bc87b4f	Make linter happy ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 15:35:06 +10:00
Charles Korn	f07b3ae67b	Fix issue where avg over mixed exponential and custom buckets, or incompatible custom buckets, produces incorrect results or panics ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 15:32:35 +10:00
Charles Korn	5ee94f49a2	Fix issue where sum over mixed exponential and custom buckets, or incompatible custom buckets, produces incorrect results ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 15:30:01 +10:00
Björn Rabenstein	ee5bba07c0	Merge pull request #14413 from prometheus/beorn7/promql ... promql: more Kahan summation (avg) and less incremental mean calculation (avg, avg_over_time)	2024-08-06 19:56:32 +02:00
György Krajcsovits	37c8c9257b	Fix histogram pool poisoning bu chunkenc.Iterator ... chunkenc.Iterator.AtFloatHistogram may do a shallow copy if it receives nil as input pointer. This can in turn share the span slice with multiple histograms in the matrixSelectorHPool, leading to unexpected errors. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-06 19:40:14 +02:00
Jan Fajerski	adf5d6bce1	Merge branch 'main' into 3.0-main-sync-24-07-18 ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com> Conflicts: VERSION pick 3.0.0 promql/promqltest/testdata/histograms.test pick changes from c39776c5b5, but adjust 5m range selectors to 10m to account for https://github.com/prometheus/prometheus/pull/13904. Fixes: promql/promqltest/testdata/functions.test promql/promqltest/testdata/staleness.test Tests added in https://github.com/prometheus/prometheus/pull/9138 need to be adjusted to account for https://github.com/prometheus/prometheus/pull/13904.	2024-07-18 15:56:40 +02:00
Arve Knudsen	fec6adadcd	Merge remote-tracking branch 'prometheus/main' into arve/close-engine	2024-07-14 13:19:11 +02:00
beorn7	c46074f4dd	promql: make avg aggregation more precise and less expensive ... The basic idea here is that the previous code was always doing incremental calculation of the mean value, which is more costly and can be less precise. It protects against overflows, but in most cases, an overflow doesn't happen anyway. The other idea applied here is to expand on #14074, where Kahan summation was applied to sum(). With this commit, the average is calculated in a conventional way (adding everything up and divide in the end) as long as the sum isn't overflowing float64. This is combined with Kahan summation so that the avg aggregation, in most cases, is really equivalent to the sum aggregation with a following division (which is the user's expectation as avg is supposed to be syntactic sugar for sum with a following divison). If the sum hits ±Inf, the calculation reverts to incremental calculation of the mean value. Kahan summation is also applied here, although it cannot fully compensate for the numerical errors introduced by the incremental mean calculation. (The tests added in this commit would fail if incremental mean calculation was always used.) Signed-off-by: beorn7 <beorn@grafana.com>	2024-07-10 19:20:24 +02:00
Filip Petkovski	acb6c1ae4b	Fix decoding buckets for native histograms in binops ... The optimizer which detects cases where histogram buckets can be skipped does not take into account binary expressions. This can lead to buckets not being decoded if a metric is used with both histogram_fraction/quantile and histogram_sum/count in the same expression. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2024-07-10 11:55:29 +02:00
Jan Fajerski	c0d67fd845	Merge branch 'main' into 3.0-main-sync-24-07-09	2024-07-09 16:51:07 +02:00
beorn7	44d8c1d182	nit: add period at end of sentence ... Signed-off-by: beorn7 <beorn@grafana.com>	2024-07-04 17:31:39 +02:00
beorn7	9a837b7f3c	promql: Make groupedAggregation.groupCount a float64 ... It's always used as such. Let's avoid the countless conversions. Signed-off-by: beorn7 <beorn@grafana.com>	2024-07-04 17:31:34 +02:00
JuanJo Ciarlante	c94c5b64c3	feat: add limitk() and limit_ratio() operators (#12503) ... * rebase 2024-07-01, picks previous renaming to `limitk()` and `limit_ratio()` Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * gofumpt -d -extra Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * more lint fixes Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * more lint fixes+ Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * put limitk() and limit_ratio() behind --enable-feature=promql-experimental-functions Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * EnableExperimentalFunctions for TestConcurrentRangeQueries() also Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * use testutil.RequireEqual to fix tests, WIP equivalent thingie for require.Contains Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * lint fix Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * moar linting Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * rebase 2024-06-19 Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * re-add limit(2, metric) testing for N=2 common series subset Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * move `ratio = param` to default switch case, for better readability Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * gofumpt -d -extra util/testutil/cmp.go Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * early break when reaching k elems in limitk(), should have always been so (!) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * small typo fix Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * no-change small break-loop rearrange for readability Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * remove IsNan(ratio) condition in switch-case, already handled as input validation Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * no-change adding some comments Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * no-change simplify fullMatrix() helper functions used for tests Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add `limitk(-1, metric)` testcase, which is handled as any k < 1 case Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * engine_test.go: no-change create `requireCommonSeries() helper func (moving code into it) for readability Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * rebase 2024-06-21 Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * engine_test.go: HAPPY NOW about its code -> reorg, create and use simpleRangeQuery() function, less lines and more readable ftW \o/ Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * move limitk(), limit_ratio() testing to promql/promqltest/testdata/limit.test Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * remove stale leftover after moving tests from engine_test.go to testdata/ Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * fix flaky `limit_ratio(0.5, ...)` test case Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * Update promql/engine.go Co-authored-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * Update promql/engine.go Co-authored-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * Update promql/engine.go Co-authored-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * fix AddRatioSample() implementation to use a single conditional (instead of switch/case + fallback return) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * docs/querying/operators.md: document r < 0 Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add negative limit_ratio() example to docs/querying/examples.md Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * move more extensive docu examples to docs/querying/operators.md Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * typo Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * small docu fix for poor-mans-normality-check, add it to limit.test ;) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * limit.test: expand "Poor man's normality check" to whole eval range Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * restore mistakenly removed existing small comment Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * expand poors-man-normality-check case(s) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * Revert "expand poors-man-normality-check case(s)" This reverts commit f69e1603b2ebe69c0a100197cfbcf6f81644b564, indeed too flaky 0:) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * remove humor from docs/querying/operators.md Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * fix signoff Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add web/ui missing changes Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * expand limit_ratio test cases, cross-fingering they'll not be flaky Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * remove flaky test Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add missing warnings.Merge(ws) in instant-query return shortcut Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add missing LimitK||LimitRatio case to codemirror-promql/src/parser/parser.ts Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * fix ui-lint Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * actually fix returned warnings :] Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> --------- Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> Co-authored-by: Julius Volz <julius.volz@gmail.com>	2024-07-03 22:18:57 +02:00
Jan Fajerski	b4152309a4	Merge branch 'main' into 3.0-main-sync ... Conflicts: promql/engine_test.go Resolved by picking main changes but adjusting total_samples for query "max_over_time(metricWith1HistogramEvery10Seconds[60s])[20s:5s]" to 312. Via https://github.com/prometheus/prometheus/pull/13662 this histogram now stores 13 values per timestamp, but via https://github.com/prometheus/prometheus/pull/13904 the range query is now left-open. promql/promqltest/testdata/functions.test Resolved by picking changes in main. See also https://github.com/prometheus/prometheus/pull/13662, but adjust some range selectors (`s/1m/2m/`) to account for https://github.com/prometheus/prometheus/pull/13904. promql/promqltest/testdata/histograms.test Resolved by picking changes in main. See also https://github.com/prometheus/prometheus/pull/13662, but adjust some range selectors (`s/5m/10m/`) to account for https://github.com/prometheus/prometheus/pull/13904. Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-07-03 09:45:25 +02:00
Arve Knudsen	e8ae8cf012	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-07-01 10:47:21 +02:00
Björn Rabenstein	2e58d46522	Merge pull request #13662 from prometheus/nhcb ... Native histograms custom buckets storage	2024-06-27 21:44:20 +02:00
Bryan Boreham	b6aba4ff14	Merge pull request #14074 from bboreham/kahan-sum-sum ... [ENHANCEMENT] PromQL: use Kahan summation for sum()	2024-06-24 11:13:26 +01:00
Zhang Zhanpeng	debbdb8608	make matrix selection and lookback left-open and right-closed ... Signed-off-by: Zhang Zhanpeng <zhangzhanpeng.zzp@alibaba-inc.com> Signed-off-by: beorn7 <beorn@grafana.com> Co-authored-by: beorn7 <beorn@grafana.com>	2024-06-20 22:05:40 +08:00
Arve Knudsen	b7320ef636	Merge remote-tracking branch 'prometheus/main' into arve/close-engine	2024-06-14 10:51:35 +02:00
Jeanette Tan	14f8dded39	Merge branch 'main' into nhcb ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-06-07 19:17:14 +08:00
Filip Petkovski	6e68046c25	Implement histogram statistics decoder (#14097) ... Implement histogram statistics decoder This commit speeds up histogram_count and histogram_sum functions on native histograms. The idea is to have separate decoders which can be used by the engine to only read count/sum values from histogram objects. This should help with reducing allocations when decoding histograms, as well as with speeding up aggregations like sum since they will be done on floats and not on histogram objects. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> --------- Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>	2024-06-06 17:17:13 +02:00
Arve Knudsen	e57aac8084	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-06-05 11:37:44 +02:00
Arve Knudsen	0cc99e677a	promql.Engine: Add Close method ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-05-28 12:01:47 +02:00
Jeanette Tan	f028496133	Merge branch 'main' into nhcb ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-05-14 16:20:15 +08:00
Charles Korn	0e934dba8e	Capture timing information while sorting ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-05-13 19:47:18 +10:00
Charles Korn	036c87223c	Ensure series in matrix values returned for instant queries are always sorted ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-05-13 11:03:15 +10:00
Bryan Boreham	ea82b49c33	[ENHANCEMENT] PromQL: use Kahan summation for sum() ... This can give a more precise result, by keeping a separate running compensation value to accumulate small errors. See https://en.wikipedia.org/wiki/Kahan_summation_algorithm Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-05-09 14:29:38 +01:00
Bryan Boreham	3fd24d1cd7	Merge pull request #13999 from bboreham/extract-promqltest ... [Test] Extract most PromQL test code into separate packages	2024-05-09 13:23:11 +01:00
Bryan Boreham	e7c77f7b40	promql: export NewTestQuery ... So that tests can call it from another package. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-05-08 16:08:04 +01:00
Jeanette Tan	796b1bbfde	Merge branch 'main' into nhcb ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-05-08 19:11:39 +08:00
Arve Knudsen	a25160e6a4	[REFACTOR] PromQL: simplify rangeEvalTimestampFunctionOverVectorSelector (#14021) ... The function `rangeEvalTimestampFunctionOverVectorSelector` appeared to be checking histogram size, however the value it used was always 0 due to subtle variable shadowing. However we don't need to pass sample values to the `timestamp` function, since the latter only cares about timestamps. This also affects peak sample count in statistics, since we are no longer copying histogram samples. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-05-08 10:39:44 +01:00
György Krajcsovits	bcafa5f1f9	Merge remote-tracking branch 'upstream/main' into update-nhcb	2024-04-24 11:06:59 +02:00
Bryan Boreham	12961c6a37	promql: refactor: eliminate one 'else' ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	0ac927515b	promql: move group-seen into group struct ... Save allocating an auxilliary array. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	7499d90913	promql: remove pointer to aggregation groups ... Just allocate in one slice. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	cfbeb6681b	promql: re-use one heap for topk and bottomk ... Slightly ugly casting saves memory. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	5e3914a27c	promql: remove histogramMean from groupedAggregation ... Re-use histogramValue since we don't need them separately. Tidy up initialization. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	2cf3c9de8f	promql: store labels per-group only for count_values ... This saves memory in other kinds of aggregation. We don't need `orderedResult` in `aggregationCountValues`; the ordering is not guaranteed. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	185290a0d2	promql: pull checking of q and k out of loop ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	4584f67e17	promql: inline nextSample function ... Move Sample out of loop to reduce allocations, otherwise it escapes to the heap. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	526ce4ee7a	promql: simplify data collection in aggregations ... We don't need a Sample, just the float and histogram values. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	2f03acbafc	promql: refactor: split topk/bottomk from sum/avg/etc ... They aggregate results in different ways. topk/bottomk don't consider histograms so can simplify data collection. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	74eed67ef6	promql: refactor: pull fetching input data out of rangeEvalAgg ... This is a cleaner split of responsibilities. We now check the sample count after calling rangeEvalAgg. Changed re-use of samples to use `Clone` and `defer`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	602eb69edf	promql: refactor: extract function nextSample ... With sub-function nextValues which we shall use shortly. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	eb41e770b7	promql: refactor: extract function addToSeries ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	53a3138eeb	promql aggregations: pre-generate mapping from inputs to outputs ... So we don't have to re-create it on every time step. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	cb6c4b3092	promql: simplify k/q parameter to topk/bottomk/quantile ... Pass it as a float64 not as interface{}. Make k a simple int, since that is the parameter to make(). Pull invalid quantile warning out of the loop. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	b3bda7df4b	promql: aggregations: skip copying input to a Vector ... We can work directly from the inputMatrix on each timestep. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	c9b6c4c55a	promql: aggregations: output directly to matrix for instant queries ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	3851b74db1	promql: aggregations: skip result vector in range queries ... Adjust test to match the lower count, since samples in the vector are no longer counted. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	59548b8a0b	promql: refactor: move collection of results into aggregation() ... We don't need to check for duplicates as aggregation cannot generate them. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	bd9bdccb22	promql: refactor: simplify internal data structures ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	5f10d17cef	promql: refactor: split out aggregations over range ... The new function `rangeEvalAgg` is mostly a copy of `rangeEval`, but without `initSeries` which we don't need and inlining the callback to `aggregation()`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	e5f667537c	promql: refactor: initialize aggregation before storing in map ... This seems more consistent to me. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	29244fb841	promql: refactor: extract count_values implementation ... The existing aggregation function is very long and covers very different cases. `aggregationCountValues` is just for `count_values`, which differs from other aggregations in that it outputs as many series per group as there are values in the input. Remove the top-level switch on string parameter type; use the same `Op` check there as elswehere. Pull checking parameters out to caller, where it is only executed once. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Bryan Boreham	8e04ab6dd4	promql: refactor: extract generateGroupingLabels function ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
Julius Volz	9b7de47787	Remove unused Dmn field on EvalNodeHelper (#13877) ... https://github.com/prometheus/prometheus/pull/13446 removed the last usage of this field, but didn't remove the field. Signed-off-by: Julius Volz <julius.volz@gmail.com>	2024-04-02 18:45:46 +02:00
György Krajcsovits	a3d1a46eda	Merge branch 'main' into nhcb	2024-03-22 14:51:48 +01:00
Bartlomiej Plotka	312e3fd728	Merge pull request #13713 from charleskorn/query-engine-interface ... rules: allow using alternative PromQL engines for rule evaluation by callers using Prometheus as a lib.	2024-03-13 14:45:42 +01:00
Björn Rabenstein	af3618fd35	Merge pull request #13667 from prometheus/beorn7/promql ... Improve TestQueryStatistics and fix bugs exposed by it	2024-03-12 16:17:11 +01:00
Charles Korn	26262a1eb7	Remove unnecessary SetQueryLogger method on QueryEngine interface ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-03-12 22:01:01 +11:00
carrychair	856f6e49c8	fix function and struct name ... Signed-off-by: carrychair <linghuchong404@gmail.com>	2024-03-09 17:53:17 +08:00
Charles Korn	4e77e8e5ef	Allow using alternative PromQL engines for rule evaluation ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-03-06 14:54:33 +11:00
beorn7	f48c7a5503	promql: Add histograms to TestQueryStatistics ... Also, fix the bugs exposed by the tests. Signed-off-by: beorn7 <beorn@grafana.com>	2024-02-29 19:02:40 +01:00
beorn7	f46dd34982	promql: Add code comment ... Signed-off-by: beorn7 <beorn@grafana.com>	2024-02-29 19:02:40 +01:00
beorn7	7d364c0451	promql: remove redundant line ... Signed-off-by: beorn7 <beorn@grafana.com>	2024-02-29 19:02:40 +01:00
Björn Rabenstein	9187bcbdd5	Merge pull request #13536 from bboreham/faster-label-replace ... promql: faster range-query of label_replace and label_join	2024-02-29 17:03:00 +01:00
machine424	f477e0539a	Move from golang.org/x/exp/slices into slices now that we only support Go >= 1.21 ... Prevent adding back golang.org/x/exp/slices. Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-02-28 14:54:53 +01:00
György Krajcsovits	5d0a0a7542	Add custom buckets to native histogram model (#13592) ... * add custom buckets to native histogram model * simple copy for custom bounds * return errors for unsupported add/sub operations * add test cases for string and update appendhistogram in scrape to account for new schema * check fields which are supposed to be unused but may affect results in equals * allow appending custom buckets histograms regardless of max schema Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-02-28 14:06:43 +01:00
Bryan Boreham	22890b1eb3	PromQL: improve warning for mixed values in aggregations ... Aggregations discard the metric name, so don't try to include it in the error message. Add a test that generates this warning. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-02-25 19:57:06 +00:00
beorn7	86d7618d84	promql: Fix wrongly scoped range vectors ... Fixes #11708. If a range vector is fixen in time with the @ modifier, it gets still moved around for different steps in a range query. Since no additional points are retrieved from the TSDB, this leads to steadily emptying the range, leading to the weird behavior described in isse #11708. This only happens for functions listed in `AtModifierUnsafeFunctions`, and the only of those that takes a range vector is `predict_linear`, which is the reason why we see it only for this particular function. Signed-off-by: beorn7 <beorn@grafana.com>	2024-02-07 18:07:51 +01:00
Bryan Boreham	d3c1f0d8e0	promql: can now remove regex field from EvalNodeHelper ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-02-04 11:32:26 +01:00
Bryan Boreham	fdd5b85e06	promql: faster range-query of label_replace and label_join ... These functions act on the labels only, so don't need to go step by step over the samples in a range query. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-02-04 11:12:55 +01:00
Alan Protasio	c006c57efc	Proposal to improve FPointSlice and HPointSlice allocation. (#13448) ... * Reusing points slice from previous series when the slice is under utilized * Adding comments on the bench test Signed-off-by: Alan Protasio <alanprot@gmail.com>	2024-02-01 16:22:38 +00:00
Bryan Boreham	74b73d1e2c	Labels: Add DropMetricName function, used in PromQL (#13446) ... This function is called very frequently when executing PromQL functions, and we can do it much more efficiently inside Labels. In the common case that `__name__` comes first in the labels, we simply re-point to start at the next label, which is nearly free. `DropMetricName` is now so cheap I removed the cache - benchmarks show everything still goes faster. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-01-25 11:48:49 +01:00
Filip Petkovski	583f3e587c	Optimize histogram iterators (#13340) ... Optimize histogram iterators Histogram iterators allocate new objects in the AtHistogram and AtFloatHistogram methods, which makes calculating rates over long ranges expensive. In #13215 we allowed an existing object to be reused when converting an integer histogram to a float histogram. This commit follows the same idea and allows injecting an existing object in the AtHistogram and AtFloatHistogram methods. When the injected value is nil, iterators allocate new histograms, otherwise they populate and return the injected object. The commit also adds a CopyTo method to Histogram and FloatHistogram which is used in the BufferedIterator to overwrite items in the ring instead of making new copies. Note that a specialized HPoint pool is needed for all of this to work (`matrixSelectorHPool`). --------- Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2024-01-23 17:02:14 +01:00
Ben Ye	d778591fd3	add more context cancellation check at evaluation time ... Signed-off-by: Ben Ye <benye@amazon.com>	2024-01-21 14:19:39 -08:00
Björn Rabenstein	bfbb13cf36	Merge pull request #13267 from linasm/simplify-native-histogram-math ... promql: simplify Native Histogram arithmetics	2024-01-18 13:50:59 +01:00
zenador	72a8f1084b	Restore more efficient version of NewPossibleNonCounterInfo annotation (#13022) ... Restore more efficient version of NewPossibleNonCounterInfo annotation Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-01-16 09:54:16 +01:00
Filip Petkovski	0e1ae1d1ca	Add comment ... Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2023-12-25 11:41:07 +01:00
Filip Petkovski	5df3820c7a	Copy last histogram point ... Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2023-12-25 11:20:51 +01:00
Filip Petkovski	1f69dcfa6b	Fix reusing float histograms ... In https://github.com/prometheus/prometheus/pull/13276 we started reusing float histogram objects to reduce allocations in PromQL. That PR introduces a bug where histogram pointers gets copied to the beginning of the histograms slice, but are still kept in the end of the slice. When a new histogram is read into the last element, it can overwrite a previous element because the pointer is the same. This commit fixes the issue by moving outdated points to the end of the slice so that we don't end up with duplicate pointers in the same buffer. In other words, the slice gets rotated so that old objects can get reused. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2023-12-14 11:53:58 +01:00
Filip Petkovski	bb8363dbb3	Add comment on SampleRingIterator ... Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2023-12-13 08:30:02 +01:00
Filip Petkovski	e2a9f8ac0f	Reuse float histogram objects ... This commit reduces the memory needed to query native histogram objects by reusing existing HPoint instances. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2023-12-11 08:24:58 +01:00
Linas Medziunas	7319ad6a0b	promql: simplify Native Histogram arithmetics ... Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>	2023-12-08 10:59:00 +02:00
beorn7	0eb0ca42c5	Update “conventional histogram” → “classic histogram” ... Signed-off-by: beorn7 <beorn@grafana.com>	2023-11-29 15:22:58 +01:00
Dimitar Dimitrov	9e3df532d8	Export promql.Engine.FindMinMaxTime ... This function is useful to analyze promQL queries. We want to use this in Mimir to record the time range which the query touches. I also chose to remove the `Engine` receiver because it was unnecessary, and it makes it easier to use, but happy to refactor that if you disagree. The function is untested on its own. If you prefer to have unit tests now that its exported, I can look into adding some. Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>	2023-11-02 13:17:35 +01:00
Julien Pivotto	f568221610	Merge pull request #13057 from prometheus/release-2.48 ... Merge release-2.48 back into main	2023-10-31 15:24:39 -04:00
Oleksandr Redko	8e5f0387a2	ci(lint): enable nolintlint and remove redundant comments (#12926) ... Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>	2023-10-31 12:35:13 +01:00
Bryan Boreham	49c5e7afe1	PromQL: reduce garbage in range-query evaluation ... The temporary variable was allocated on the heap, and it is unnecessary. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2023-10-29 19:45:06 +00:00