101 commits

Author	SHA1	Message	Date
Neeraj Gartia	20371118b6	[FIX] PromQL: Ignore histograms in scalar, sort and sort_desc functions (#15964) ... PromQL: Ignore histograms in scalar, sort and sort_desc functions --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2025-02-06 16:02:03 +01:00
Björn Rabenstein	cb096a8ea8	Merge pull request #15902 from prometheus/beorn7/promql ... promql: fix rate calculation with a counter reset after the 1st sample	2025-02-05 22:10:45 +01:00
Björn Rabenstein	fa1bd02c99	Merge pull request #15974 from jhesketh/jhesketh/utf-label-replace ... Allow UTF-8 labels in label_replace	2025-02-05 20:49:36 +01:00
Joshua Hesketh	5a5fdea7ad	Fix duplicate output vector if delayed name removal is disabled (#15975) ... Fix duplicate output vector if delayed name removal is disabled This error is emitted in cleanupMetricLabels, but is skipped if enableDelayedNameRemoval is false. This makes it consistent with label_replace Signed-off-by: Joshua Hesketh <josh@nitrotech.org> --------- Signed-off-by: Joshua Hesketh <josh@nitrotech.org> Signed-off-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2025-02-05 16:40:23 +01:00
Joshua Hesketh	5be3197701	Update the test to invalid utf-8 ... "invalid-label-name" is valid in utf-8 mode Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	2025-02-05 23:08:24 +11:00
Neeraj Gartia	130cd024e0	[FEATURE] PromQL: Implements idelta and irate with histograms (#15853) ... Add native histogram support to idelta and irate functions --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2025-01-31 23:11:03 +01:00
beorn7	2581c7d057	promql: fix rate calculation with a counter reset after the 1st histogram ... If a rate (or increase) is calculated on native histograms, and there is a counter reset between the 1st and 2nd histogram, we never have to touch the 1st histogram, so it doesn't even matter if it has an incompatible bucket layout. So we should not error out in that case. This simply nulls out the 1st histogram in that case. Signed-off-by: beorn7 <beorn@grafana.com>	2025-01-29 15:42:47 +01:00
Neeraj Gartia	b9fcc81695	adds tests for timestamp() ... Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2025-01-28 02:10:36 +05:30
Neeraj Gartia	21afc0beb4	adds tests for sum_over_time and avg_over_time ... Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2025-01-28 02:05:30 +05:30
György Krajcsovits	9097f8f4e7	test(promql): some functions silently ignore native histograms ... Functions that silently ignore native histograms, not covered by existing tests: abs(), ceil(), clamp(), clamp_min(), clamp_max(), floor(), round(). Ref: https://github.com/prometheus/prometheus/pull/15845 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2025-01-24 11:16:20 +01:00
Fiona Liao	f46b984dd1	Add additional incompatible nhcb schemas tests for functions and comparison operators (#15813) ... promql: Add additional incompatible nhcb schemas tests for functions and comparison operators * Add agg_over_time tests for nhcb with incompatible schemas * Add more function and comparison operator tests --------- Signed-off-by: Fiona Liao <fiona.liao@grafana.com>	2025-01-17 17:17:11 +01:00
Neeraj Gartia	b3e30d52ce	[BUGFIX] PromQL: Fix <aggr_over_time> functions with histograms (#15711) ... fix aggr_over_time with histograms Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2025-01-09 16:38:42 +01:00
Fiona Liao	9d6f88cb73	Add additional tests for operators over incompatible nhcb (#15787) ... * Add additional tests for operators over incompatible nhcb Signed-off-by: Fiona Liao <fiona.liao@grafana.com>	2025-01-09 10:29:57 +01:00
beorn7	f7373a1f91	PromQL: improve some test comments ... This amends #15686. Signed-off-by: beorn7 <beorn@grafana.com>	2024-12-19 23:56:37 +01:00
Neeraj Gartia	8d5236f927	PromQL: Adds tests for delta with histograms (#15674) ... PromQL: Adds tests for delta with histograms Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> -------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-12-19 23:52:26 +01:00
Neeraj Gartia	0e99ca3e8c	[BUGFIX] PromQL: Fix deriv, predict_linear and double_exponential_smoothing with histograms (#15686) ... PromQL: Fix deriv, predict_linear and double_exponential_smoothing with histograms Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-12-19 23:50:28 +01:00
Neeraj Gartia	9009724c5b	[BUGFIX] PromQL: Adds test for absent, absent_over_time and deriv func with histograms (#15667) ... Add test for `absent`, `absent_over_time` and `deriv` func with histograms Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-12-13 13:34:55 +01:00
Neeraj Gartia	7fd3b13bd2	[TEST] PromQL: Adds tests with histograms for simple functions (#15488) ... adds tests with histograms for simple functions Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-12-04 13:11:05 +01:00
Neeraj Gartia	4aeed2c4b1	fix resets function for histograms ... Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-12-03 19:01:05 +05:30
Björn Rabenstein	dfbfe8b5e8	Merge pull request #15479 from NeerajGartia21/promql/datewrapper ... [BUGFIX] PromQL: Ignore histograms in all time related functions	2024-11-29 00:26:52 +01:00
Neeraj Gartia	0de340c1b4	ignore histogram in dateWrapper ... Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-11-28 01:57:04 +05:30
Neeraj Gartia	36e0897f0f	[BUGFIX] PromQL: Fix behaviour of changes() for mix of histograms and floats (#15469) ... PromQL: Fix behaviour of changes() for mix of histograms and floats --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-11-27 21:05:08 +01:00
beorn7	a44c449155	promql: Add NaN tests for avg and avg_over_time ... Signed-off-by: beorn7 <beorn@grafana.com>	2024-11-27 13:08:34 +01:00
Neeraj Gartia	38bb6ece25	[BUGFIX] PromQL: Fix behaviour of some aggregations with histograms (#15432) ... promql: fix some aggregations for histograms This PR fixes the behaviour of `topk`,`bottomk`, `limitk` and `limit_ratio` with histograms. The fixed behaviour are as follows: - For `topk` and `bottomk` histograms are ignored and add info annotations added. - For `limitk` and `limit_ratio` histograms are included in the results(if applicable). Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-11-26 19:12:36 +01:00
Björn Rabenstein	125a90899c	promqltest: Complete the tests for info annotations (#15429) ... promqltest: Complete the tests for info annotations So far, we did not test for the _absence_ of an info annotation (because many tests triggered info annotations, which we haven't taken into account so far). The test for info annotations was also missed for range queries. This completes the tests for info annotations (and refactors the many `if` statements into a somewhat more compact `switch` statement). It fixes most tests to not emit an info annotation anymore. Or it changes the `eval` to `eval_info` where we actually want to test for the info annotation. It also fixes a few spelling errors in comments. --------- Signed-off-by: beorn7 <beorn@grafana.com> Signed-off-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-11-21 14:20:38 +01:00
Neeraj Gartia	a6fb16fcb4	PromQL: Convert more native histogram tests to promql-test framework (#15419) ... This converts `TestNativeHistogram_SubOperator` to the promql testing framework. It also removes `TestNativeHistogram_Sum_Count_Add_AvgOperator`, which got converted earlier. Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-11-20 11:41:36 +01:00
Björn Rabenstein	4ef1170868	Merge pull request #15422 from NeerajGartia21/promql-corrections ... [BUGFIX] PromQL: Fix `count_values` for histograms	2024-11-20 11:27:32 +01:00
George Krajcsovits	5cd9855999	tests(promql/testdata): add regression test for and-on (#15425) ... * tests(promql/testdata): add regression test for and-on I'd like to use queries of the form "x and on() (vector(y)==1)" to be able to include and exclude series for dashboards. This helps migration to native histograms in dashboards by using a dashboard variable to set "y" to either -1 or 1 to exclude or include the result. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-11-20 10:29:18 +01:00
Neeraj Gartia	048222867a	fix count_values for histograms ... Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-11-20 02:07:31 +05:30
Charles Korn	62e6e55c07	promql: fix issues with comparison binary operations with bool modifier and native histograms (#15413) ... * Fix issue where comparison operations with `bool` modifier and native histograms return histograms rather than 0 or 1 * Don't emit anything for comparisons between floats and histograms when `bool` modifier is used * Don't emit anything for comparisons between floats and histograms when `bool` modifier is used between a vector and a scalar --------- Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-11-19 09:13:34 +01:00
Neeraj Gartia	789c9b1a5e	[BUGFIX] PromQL: Corrects the behaviour of some operator and aggregators with Native Histograms (#15245) ... PromQL: Correct the behaviour of some operator and aggregators with Native Histograms --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-11-12 15:37:05 +01:00
Joshua Hesketh	ed2668bbda	Merge branch 'main' into jhesketh/clamp ... Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	2024-11-12 10:20:58 +11:00
Bryan Boreham	6979b237b9	Merge branch 'release-2.55' into merge-2.55-into-main2	2024-11-04 12:58:38 +00:00
Ben Ye	b7aca45de7	fix round function ignoring enableDelayedNameRemoval feature flag ... Signed-off-by: Ben Ye <benye@amazon.com>	2024-10-31 00:30:22 -07:00
Joshua Hesketh	14ef1ce8ab	Round function should ignore native histograms ... As per the documentation, native histograms are skipped. This is in line with other simpleFunc's. Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	2024-10-17 15:39:48 +11:00
Joshua Hesketh	5a4e4f6936	Fix stddev/stdvar when aggregating histograms, NaNs, and infinities (#14941) ... promql: Fix stddev/stdvar when aggregating histograms, NaNs, and Infs Native histograms are ignored when calculating stddev or stdvar. However, for the first series of each group, a `groupedAggregation` is always created. If the first series that was encountered is a histogram then it acts as the equivalent of a 0 point. This change creates the first `groupedAggregation` with the `seen` field set to `false` if the point is a histogram, thus ignoring it like the rest of the aggregation function does. A new `groupedAggregation` will then be created once an actual float value is encountered. This commit also sets the `floatValue` field of the `groupedAggregation` to `NaN`, if the first float value of a group is `NaN` or `±Inf`, so that the outcome is consistently `NaN` once those values are in the mix. (The added tests fail without this change). Signed-off-by: Joshua Hesketh <josh@nitrotech.org> Signed-off-by: beorn7 <beorn@grafana.com> --------- Signed-off-by: Joshua Hesketh <josh@nitrotech.org> Signed-off-by: beorn7 <beorn@grafana.com> Co-authored-by: beorn7 <beorn@grafana.com>	2024-10-16 15:00:46 +02:00
Joshua Hesketh	31d19381f6	Clamp functions should ignore native histograms ... As per the documentation, native histograms are skipped. This is in line with other `simpleFunc`'s. Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	2024-10-16 15:10:54 +11:00
Neeraj Gartia	d4b1f9eb33	Corrects the behaviour of binary opperators between histogram and float (#14726) ... promql: corrects binary operators functioning for mixed sample with histogram and float For invalid pairings of sample types, an annotation is added now. Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-10-15 14:44:36 +02:00
Fiona Liao	8650d25804	Add additional basic nhcb unit tests (#15086) ... * Add additional basic nhcb unit tests * Update promql/promqltest/testdata/histograms.test Signed-off-by: Fiona Liao <fiona.liao@grafana.com> Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2024-10-08 14:34:32 +02:00
Björn Rabenstein	df9916ef66	Merge pull request #14677 from prometheus/beorn7/histogram ... promql(native histograms): Introduce exponential interpolation	2024-09-19 18:08:59 +02:00
Jan Fajerski	96e5a94d29	promql: rename holt_winters to double_exponential_smoothing ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-09-19 15:29:01 +02:00
Björn Rabenstein	1639450172	Merge pull request #14821 from charleskorn/nh-negative-multiplication-division ... promql: correctly handle unary negation of native histograms and add tests for multiplication and division of native histograms by negative scalars Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-09-19 14:07:37 +01:00
beorn7	6fcd225aee	promql(native histograms): Introduce exponential interpolation ... The linear interpolation (assuming that observations are uniformly distributed within a bucket) is a solid and simple assumption in lack of any other information. However, the exponential bucketing used by standard schemas of native histograms has been chosen to cover the whole range of observations in a way that bucket populations are spread out over buckets in a reasonably way for typical distributions encountered in real-world scenarios. This is the origin of the idea implemented here: If we divide a given bucket into two (or more) smaller exponential buckets, we "most naturally" expect that the samples in the original buckets will split among those smaller buckets in a more or less uniform fashion. With this assumption, we end up with an "exponential interpolation", which therefore appears to be a better match for histograms with exponential bucketing. This commit leaves the linear interpolation in place for NHCB, but changes the interpolation for exponential native histograms to exponential. This affects `histogram_quantile` and `histogram_fraction` (because the latter is more or less the inverse of the former). The zero bucket has to be treated specially because the assumption above would lead to an "interpolation to zero" (the bucket density approaches infinity around zero, and with the postulated uniform usage of buckets, we would end up with an estimate of zero for all quantiles ending up in the zero bucket). We simply fall back to linear interpolation within the zero bucket. At the same time, this commit makes the call to stick with the assumption that the zero bucket only contains positive observations for native histograms without negative buckets (and vice versa). (This is an assumption relevant for interpolation. It is a mostly academic point, as the zero bucket is supposed to be very small anyway. However, in cases where it _is_ relevantly broad, the assumption helps a lot in practice.) This commit also updates and completes the documentation to match both details about interpolation. As a more high level note: The approach here attempts to strike a balance between a more simplistic approach without any assumption, and a more involved approach with more sophisticated assumptions. I will shortly describe both for reference: The "zero assumption" approach would be to not interpolate at all, but _always_ return the harmonic mean of the bucket boundaries of the bucket the quantile ends up in. This has the advantage of minimizing the maximum possible relative error of the quantile estimation. (Depending on the exact definition of the relative error of an estimation, there is also an argument to return the arithmetic mean of the bucket boundaries.) While limiting the maximum possible relative error is a good property, this approach would throw away the information if a quantile is closer to the upper or lower end of the population within a bucket. This can be valuable trending information in a dashboard. With any kind of interpolation, the maximum possible error of a quantile estimation increases to the full width of a bucket (i.e. it more than doubles for the harmonic mean approach, and precisely doubles for the arithmetic mean approach). However, in return the _expectation value_ of the error decreases. The increase of the theoretical maximum only has practical relevance for pathologic distributions. For example, if there are thousand observations within a bucket, they could _all_ be at the upper bound of the bucket. If the quantile calculation picks the 1st observation in the bucket as the relevant one, an interpolation will yield a value close to the lower bucket boundary, while the true quantile value is close to the upper boundary. The "fancy interpolation" approach would be one that analyses the _actual_ distribution of samples in the histogram. A lot of statistics could be applied based on the information we have available in the histogram. This would include the population of neighboring (or even all) buckets in the histogram. In general, the resolution of a native histogram should be quite high, and therefore, those "fancy" approaches would increase the computational cost quite a bit with very little practical benefits (i.e. just tiny corrections of the estimated quantile value). The results are also much harder to reason with. Signed-off-by: beorn7 <beorn@grafana.com>	2024-09-19 14:19:10 +02:00
Jan Fajerski	91608c002f	Merge branch 'main' into release-3.0-beta.0 ... Conflicts: scrape/scrape_test.go Pick both changes.	2024-09-10 20:51:20 +02:00
Charles Korn	e8c7482137	Return negative counts when multiplied or divided by a negative value ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-09 14:37:59 +10:00
Charles Korn	113de6301c	Add failing test cases for unary negation and multiplication and division with negative scalars ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-04 16:20:28 +10:00
Charles Korn	9b451abec7	Make positive and negative bucket counts different in existing test cases ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-04 16:08:05 +10:00
Jan Fajerski	956245b25b	promqltest: adjust eval times and range selector ... In order to fix new tests for changes added in https://github.com/prometheus/prometheus/pull/13904. Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-09-02 11:27:39 +02:00
Jan Fajerski	00315ce15e	Merge branch 'main' into 3.0-main-sync-24-08-30 ... using -Xours Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-09-02 11:27:18 +02:00
Neeraj Gartia	8c7bf39d96	Moves TestNativeHistogram_MulDivOperator to promql testing framework (#14688) ... PromQL: add test for mul and div operator Also, remove the converted test from the engine_test.go file. This also includes an extension of the test framework to allow NaN/Inf in histogram buckets. --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-08-29 16:42:35 +02:00