202 commits

Author	SHA1	Message	Date
Arve Knudsen	db5e48dc33	promql.Engine.Close: No-op if nil (#14861) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-09-08 14:39:13 +02:00
Arve Knudsen	5dfbcc390e	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-09-02 16:26:59 +02:00
Neeraj Gartia	8c7bf39d96	Moves TestNativeHistogram_MulDivOperator to promql testing framework (#14688) ... PromQL: add test for mul and div operator Also, remove the converted test from the engine_test.go file. This also includes an extension of the test framework to allow NaN/Inf in histogram buckets. --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-08-29 16:42:35 +02:00
Jorge Creixell	e9e3d64b7c	PromQL engine: Delay deletion of __name__ label to the end of the query evaluation (#14477) ... PromQL engine: Delay deletion of __name__ label to the end of the query evaluation - This change allows optionally preserving the `__name__` label via the `label_replace` and `label_join` functions, and helps prevent the dreaded "vector cannot contain metrics with the same labelset" error. - The implementation extends the `Series` and `Sample` structs with a boolean flag indicating whether the `__name__` label should be deleted at the end of the query evaluation. - The `label_replace` and `label_join` functions can still access the value of the `__name__` label, even if it has been previously marked for deletion. If `__name__` is used as target label, it won't be dropped at the end of the query evaluation. - Fixes https://github.com/prometheus/prometheus/issues/11397 - See https://github.com/jcreixell/prometheus/pull/2 for previous discussion, including the decision to create this PR and benchmark it before considering other alternatives (like refactoring `labels.Labels`). - See https://github.com/jcreixell/prometheus/pull/1 for an alternative implementation using a special label instead of boolean flags. - Note: a feature flag `promql-delayed-name-removal` has been added as it changes the behavior of some "weird" queries (see https://github.com/prometheus/prometheus/issues/11397#issuecomment-1451998792) Example (this always fails, as `__name__` is being dropped by `count_over_time`): ``` count_over_time({__name__!=""}[1m]) => Error executing query: vector cannot contain metrics with the same labelset ``` Before: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => Error executing query: vector cannot contain metrics with the same labelset ``` After: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => count_go_gc_cycles_automatic_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 count_go_gc_cycles_forced_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 ... ``` Signed-off-by: Jorge Creixell <jcreixell@gmail.com> --------- Signed-off-by: Jorge Creixell <jcreixell@gmail.com> Signed-off-by: Björn Rabenstein <github@rabenste.in>	2024-08-29 15:50:39 +02:00
Arve Knudsen	99204f23ee	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-29 09:52:54 +02:00
Arve Knudsen	c9a460d570	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-26 12:17:10 +02:00
Ziqi Zhao	8f828d45c1	convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-21 09:24:50 +08:00
György Krajcsovits	b91acc61b0	Add unit test ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-06 20:50:51 +02:00
Max Amin	84b819a69f	feat: add Google cloud roundtripper for remote write (#14346) ... * feat: Google Auth for remote write Signed-off-by: Max Amin <maxamin@google.com> --------- Signed-off-by: Max Amin <maxamin@google.com>	2024-07-30 16:25:19 +01:00
Arve Knudsen	fbc9eddfaf	Refactor engine creation in tests ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-07-14 13:58:51 +02:00
Arve Knudsen	fec6adadcd	Merge remote-tracking branch 'prometheus/main' into arve/close-engine	2024-07-14 13:19:11 +02:00
Bryan Boreham	d116bf7b9f	Merge pull request #14109 from harry671003/pass_limit_to_querier ... storage: pass limit param as hint in querier	2024-07-12 10:27:52 +01:00
JuanJo Ciarlante	c94c5b64c3	feat: add limitk() and limit_ratio() operators (#12503) ... * rebase 2024-07-01, picks previous renaming to `limitk()` and `limit_ratio()` Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * gofumpt -d -extra Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * more lint fixes Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * more lint fixes+ Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * put limitk() and limit_ratio() behind --enable-feature=promql-experimental-functions Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * EnableExperimentalFunctions for TestConcurrentRangeQueries() also Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * use testutil.RequireEqual to fix tests, WIP equivalent thingie for require.Contains Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * lint fix Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * moar linting Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * rebase 2024-06-19 Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * re-add limit(2, metric) testing for N=2 common series subset Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * move `ratio = param` to default switch case, for better readability Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * gofumpt -d -extra util/testutil/cmp.go Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * early break when reaching k elems in limitk(), should have always been so (!) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * small typo fix Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * no-change small break-loop rearrange for readability Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * remove IsNan(ratio) condition in switch-case, already handled as input validation Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * no-change adding some comments Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * no-change simplify fullMatrix() helper functions used for tests Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add `limitk(-1, metric)` testcase, which is handled as any k < 1 case Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * engine_test.go: no-change create `requireCommonSeries() helper func (moving code into it) for readability Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * rebase 2024-06-21 Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * engine_test.go: HAPPY NOW about its code -> reorg, create and use simpleRangeQuery() function, less lines and more readable ftW \o/ Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * move limitk(), limit_ratio() testing to promql/promqltest/testdata/limit.test Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * remove stale leftover after moving tests from engine_test.go to testdata/ Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * fix flaky `limit_ratio(0.5, ...)` test case Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * Update promql/engine.go Co-authored-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * Update promql/engine.go Co-authored-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * Update promql/engine.go Co-authored-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * fix AddRatioSample() implementation to use a single conditional (instead of switch/case + fallback return) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * docs/querying/operators.md: document r < 0 Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add negative limit_ratio() example to docs/querying/examples.md Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * move more extensive docu examples to docs/querying/operators.md Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * typo Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * small docu fix for poor-mans-normality-check, add it to limit.test ;) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * limit.test: expand "Poor man's normality check" to whole eval range Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * restore mistakenly removed existing small comment Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * expand poors-man-normality-check case(s) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * Revert "expand poors-man-normality-check case(s)" This reverts commit f69e1603b2ebe69c0a100197cfbcf6f81644b564, indeed too flaky 0:) Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * remove humor from docs/querying/operators.md Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * fix signoff Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add web/ui missing changes Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * expand limit_ratio test cases, cross-fingering they'll not be flaky Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * remove flaky test Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add missing warnings.Merge(ws) in instant-query return shortcut Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * add missing LimitK||LimitRatio case to codemirror-promql/src/parser/parser.ts Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * fix ui-lint Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> * actually fix returned warnings :] Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> --------- Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com> Co-authored-by: Julius Volz <julius.volz@gmail.com>	2024-07-03 22:18:57 +02:00
Arve Knudsen	493e196f10	Fix bad merge ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-07-03 15:12:40 +02:00
Arve Knudsen	e8ae8cf012	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-07-01 10:47:21 +02:00
🌲 Harry 🌊 John 🏔	d5f6887294	Pass limit param as hint to storage.Querier ... Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>	2024-06-20 09:47:38 -07:00
Jeanette Tan	dda5f48c9e	Merge branch 'main' into nhcb-review-2	2024-06-20 22:50:00 +08:00
Charles Korn	aeec30f082	Convert TestTimestampFunction_StepsMoreOftenThanSamples ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-06-17 16:56:56 +10:00
Charles Korn	987fa5c6a2	Convert range query test cases to test scripting language ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-06-17 16:43:01 +10:00
Jeanette Tan	14f8dded39	Merge branch 'main' into nhcb ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-06-07 19:17:14 +08:00
Arve Knudsen	e57aac8084	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-06-05 11:37:44 +02:00
Charles Korn	24ef000025	Merge branch 'main' into sort-matrix-series ... # Conflicts: # promql/engine_test.go	2024-05-31 12:44:27 +10:00
Arve Knudsen	0cc99e677a	promql.Engine: Add Close method ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-05-28 12:01:47 +02:00
Arve Knudsen	7b56353090	Merge remote-tracking branch 'prometheus/main' into arve/query-logger-munmap	2024-05-27 17:08:33 +02:00
Björn Rabenstein	114dc5c393	Merge pull request #13638 from NeerajGartia21/promql-test ... Converts existing native histogram unit tests to the PromQL testing framework	2024-05-19 19:35:36 +02:00
Arve Knudsen	0f01d4b336	Fix flaky test ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-05-15 21:58:56 +02:00
Arve Knudsen	bf8d88f326	Merge remote-tracking branch 'origin/main' into arve/query-logger-munmap	2024-05-15 21:02:03 +02:00
Oleksandr Redko	f10c3454e9	Enable perfsprint linter and fix up code ... Signed-off-by: Oleksandr Redko <oleksandr.red+github@gmail.com>	2024-05-15 17:51:05 +03:00
Jeanette Tan	f028496133	Merge branch 'main' into nhcb ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-05-14 16:20:15 +08:00
Neeraj Gartia	661856cb65	removes the added tests from engine_test.go ... Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-05-13 22:58:25 +05:30
Charles Korn	a4381608a0	Add failing test case ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-05-13 11:03:14 +10:00
Arve Knudsen	aabe4d6e4a	promql.ActiveQueryTracker: Unmap mmapped file when done ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-05-09 14:27:29 +02:00
Bryan Boreham	3fd24d1cd7	Merge pull request #13999 from bboreham/extract-promqltest ... [Test] Extract most PromQL test code into separate packages	2024-05-09 13:23:11 +01:00
Bryan Boreham	786e0e706c	test: PromQL: stop using internal fields of engine ... * set enablePerStepStats and lookback duration via `NewTestEngine` parameters. * check maxSamples by recreating query engine * check lookback without modifying internals Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-05-08 19:58:41 +01:00
Bryan Boreham	0dbfd20b69	test: move most PromQL tests into separate test package ... So that they can import promqltest which imports promql. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-05-08 16:28:56 +01:00
Bryan Boreham	4a72607c4a	refactor: extract some PromQL Engine tests which use unexported structs ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-05-08 16:27:17 +01:00
Bryan Boreham	babfcfdd91	refactor: Move NewTestEngine into promqltest ... And export `DefaultMaxSamplesPerQuery` so callers can replicate previous behaviour. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-05-08 16:07:28 +01:00
Bryan Boreham	a1af3c27d4	refactor: extract almost.Equal() to new package ... To avoid a circular reference between promql and promqltest. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-05-08 13:42:10 +01:00
Jeanette Tan	796b1bbfde	Merge branch 'main' into nhcb ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-05-08 19:11:39 +08:00
Arve Knudsen	a25160e6a4	[REFACTOR] PromQL: simplify rangeEvalTimestampFunctionOverVectorSelector (#14021) ... The function `rangeEvalTimestampFunctionOverVectorSelector` appeared to be checking histogram size, however the value it used was always 0 due to subtle variable shadowing. However we don't need to pass sample values to the `timestamp` function, since the latter only cares about timestamps. This also affects peak sample count in statistics, since we are no longer copying histogram samples. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-05-08 10:39:44 +01:00
György Krajcsovits	bcafa5f1f9	Merge remote-tracking branch 'upstream/main' into update-nhcb	2024-04-24 11:06:59 +02:00
Bryan Boreham	3851b74db1	promql: aggregations: skip result vector in range queries ... Adjust test to match the lower count, since samples in the vector are no longer counted. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-04-05 15:47:54 +01:00
György Krajcsovits	2a4aa085d2	Merge branch 'main' into nhcb	2024-03-27 18:42:10 +01:00
George Krajcsovits	dc7b282d39	engine_test: adjust and comment histogram sample counts (#13841) ... The size of histogram points are now bigger by 24 bytes due to the custom values slice. When histograms are loaded into partial results in vector selectors we use HPoint type where the size is calculated as (size of histogram + 8 for timestamp)/16. a3d1a46eda/promql/value.go (L176) When histograms are put into Sample type in range evaluations, the Sample has more overhead and the size is calculated differently: (size of histogram / 16) + 1 for time stamp. a3d1a46eda/promql/engine.go (L1928) When the size of the histogram is 16k, then the first calculation gives k but the second gives k+1 for the sample count. If the histogram size is 16k+8, then both would give k+1. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-03-27 18:19:14 +01:00
Björn Rabenstein	b9a2a4e329	Merge pull request #13852 from prometheus/fix-hist-std-dev-var-negative ... Fix hist std dev var negative	2024-03-27 17:58:03 +01:00
Jeanette Tan	22d0f4f114	improve handling of negative bounds in histogram std dev/var ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-03-27 17:06:12 +01:00
Domantas	435f330d0b	[BUGFIX] labels: don't modify original labels in DropMetricName (#13845) ... Restrict the capacity of first argument to `append()` to force an allocation. This is for the slice implementation only. Signed-off-by: Domantas Jadenkus <djadenkus@gmail.com>	2024-03-27 10:35:17 +00:00
Jeanette Tan	9d32754bc0	add unit tests with all negative values for histogram_stddev and var ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-03-22 03:42:50 +08:00
beorn7	f48c7a5503	promql: Add histograms to TestQueryStatistics ... Also, fix the bugs exposed by the tests. Signed-off-by: beorn7 <beorn@grafana.com>	2024-02-29 19:02:40 +01:00
beorn7	f46dd34982	promql: Add code comment ... Signed-off-by: beorn7 <beorn@grafana.com>	2024-02-29 19:02:40 +01:00