1064 commits

Author	SHA1	Message	Date
Neeraj Gartia	8c7bf39d96	Moves TestNativeHistogram_MulDivOperator to promql testing framework (#14688) ... PromQL: add test for mul and div operator Also, remove the converted test from the engine_test.go file. This also includes an extension of the test framework to allow NaN/Inf in histogram buckets. --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-08-29 16:42:35 +02:00
Jorge Creixell	e9e3d64b7c	PromQL engine: Delay deletion of __name__ label to the end of the query evaluation (#14477) ... PromQL engine: Delay deletion of __name__ label to the end of the query evaluation - This change allows optionally preserving the `__name__` label via the `label_replace` and `label_join` functions, and helps prevent the dreaded "vector cannot contain metrics with the same labelset" error. - The implementation extends the `Series` and `Sample` structs with a boolean flag indicating whether the `__name__` label should be deleted at the end of the query evaluation. - The `label_replace` and `label_join` functions can still access the value of the `__name__` label, even if it has been previously marked for deletion. If `__name__` is used as target label, it won't be dropped at the end of the query evaluation. - Fixes https://github.com/prometheus/prometheus/issues/11397 - See https://github.com/jcreixell/prometheus/pull/2 for previous discussion, including the decision to create this PR and benchmark it before considering other alternatives (like refactoring `labels.Labels`). - See https://github.com/jcreixell/prometheus/pull/1 for an alternative implementation using a special label instead of boolean flags. - Note: a feature flag `promql-delayed-name-removal` has been added as it changes the behavior of some "weird" queries (see https://github.com/prometheus/prometheus/issues/11397#issuecomment-1451998792) Example (this always fails, as `__name__` is being dropped by `count_over_time`): ``` count_over_time({__name__!=""}[1m]) => Error executing query: vector cannot contain metrics with the same labelset ``` Before: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => Error executing query: vector cannot contain metrics with the same labelset ``` After: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => count_go_gc_cycles_automatic_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 count_go_gc_cycles_forced_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 ... ``` Signed-off-by: Jorge Creixell <jcreixell@gmail.com> --------- Signed-off-by: Jorge Creixell <jcreixell@gmail.com> Signed-off-by: Björn Rabenstein <github@rabenste.in>	2024-08-29 15:50:39 +02:00
Björn Rabenstein	849215d90c	Merge pull request #14585 from fatsheep9146/covert-TestNativeHistogram_Sum_Count_Add_AvgOperator-to-framework ... convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework	2024-08-28 17:21:32 +02:00
Owen Williams	9da75328ea	fix(utf8): ensure correct validation when legacy mode turned on (#14736) ... fix(utf8): ensure correct validation when legacy mode turned on This depends on the included update of the prometheus/common dependency. --------- Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-28 17:15:42 +02:00
Ziqi Zhao	87eab0aaad	fix golang lint ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-23 09:32:04 +08:00
Ziqi Zhao	c7c4a5c347	add helper function to compare native histograms in testing ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-23 09:28:31 +08:00
beorn7	0f760f63dd	lint: Revamp our linting rules, mostly around doc comments ... Several things done here: - Set `max-issues-per-linter` to 0 so that we actually see all linter warnings and not just 50 per linter. (As we also set `max-same-issues` to 0, I assume this was the intention from the beginning.) - Stop using the golangci-lint default excludes (by setting `exclude-use-default: false`. Those are too generous and don't match our style conventions. (I have re-added some of the excludes explicitly in this commit. See below.) - Re-add the `errcheck` exclusion we have used so far via the defaults. - Exclude the signature requirement `govet` has for `Seek` methods because we use non-standard `Seek` methods a lot. (But we keep other requirements, while the default excludes completely disabled the check for common method segnatures.) - Exclude warnings about missing doc comments on exported symbols. (We used to be pretty adamant about doc comments, but stopped that at some point in the past. By now, we have about 500 missing doc comments. We may consider reintroducing this check, but that's outside of the scope of this commit. The default excludes of golangci-lint essentially ignore doc comments completely.) - By stop using the default excludes, we now get warnings back on malformed doc comments. That's the most impactful change in this commit. It does not enforce doc comments (again), but _if_ there is a doc comment, it has to have the recommended form. (Most of the changes in this commit are fixing this form.) - Improve wording/spelling of some comments in .golangci.yml, and remove an outdated comment. - Leave `package-comments` inactive, but add a TODO asking if we should change that. - Add a new sub-linter `comment-spacings` (and fix corresponding comments), which avoids missing spaces after the leading `//`. Signed-off-by: beorn7 <beorn@grafana.com>	2024-08-22 17:36:11 +02:00
Björn Rabenstein	7fad1ec8ee	Merge pull request #14655 from suntala/suntala/sort-by-label-enhancement ... promql: Fall back to full label sets when sorting by label	2024-08-21 12:28:55 +02:00
Ziqi Zhao	8f828d45c1	convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-21 09:24:50 +08:00
suntala	8e82ac8d2b	Add comments to the sort by label functions ... Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-20 20:41:55 +02:00
Arve Knudsen	3a78e76282	Upgrade golangci-lint to v1.60.1 ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-18 12:13:25 +02:00
Björn Rabenstein	1daf7cdd62	Merge pull request #14626 from cuiweiyuan/main ... chore: fix some function names	2024-08-15 11:46:21 +02:00
cuiweiyuan	1800af54f0	chore: fix some function names ... Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>	2024-08-15 13:57:21 +08:00
Charles Korn	52818a97e2	Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms ... # Conflicts: # promql/promqltest/testdata/native_histograms.test	2024-08-14 07:52:08 +10:00
Björn Rabenstein	c2bc6cfe97	Merge pull request #14621 from charleskorn/panic-message ... promql: clarify error message logged when panic occurs during query evaluation	2024-08-13 23:02:43 +02:00
George Krajcsovits	fe5354df88	Merge pull request #14658 from krajorama/zero-rate-nh-quantile ... Native histograms: define behavior when rate is null.	2024-08-13 16:05:43 +02:00
György Krajcsovits	386fc8b9f6	Update from review comments. ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-13 15:26:07 +02:00
Arve Knudsen	0503d4f372	PromQL: Fix comment regarding non-nil histogram pointer ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-13 08:55:24 +02:00
György Krajcsovits	6aee5b4b38	fix typo ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-12 12:04:45 +02:00
György Krajcsovits	06a8886b94	Native histograms: define behavior when rate is null. ... Histogram quantile returns NaN in this case, which might be surprising, so add a unit test that clarifies that this is intentional. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-12 10:40:21 +02:00
suntala	fd2f44af7f	Fall back to comparing by label set when sorting by label desc ... Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com> Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-11 21:44:03 +02:00
suntala	94ad489328	Fall back to comparing by label set when sorting by label ... Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com> Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-11 21:44:03 +02:00
Charles Korn	f992f81bd0	Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms ... Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com>	2024-08-09 13:58:54 +10:00
Charles Korn	5cfdde327c	Address PR feedback: add extra test case ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-09 13:57:37 +10:00
Charles Korn	82bb35fabb	Address PR feedback: fix typo and rename variable ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-09 13:51:31 +10:00
Björn Rabenstein	41f8a0223c	Merge pull request #14609 from charleskorn/rate-over-mixed-custom-exponential-histograms ... promql: fix "cannot reduce resolution to custom buckets schema" panic in `rate` over native histograms with mix of custom and exponential buckets	2024-08-08 14:03:41 +02:00
Manik Rana	02c465bf58	textparse: Implement CreatedTimestamp() in openmetricsparse.go (#14356) ... * feat: initial implement of createedTimestamp() with tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: return ct after finding it Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove unneeded test Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: add comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: multiple changes - implement changes from pair programming session - use newParse.val() - advance parser p if ct is found Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: check if err from p.Next() Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: advance parser and parse histograms + summary Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: restore previous tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: retore failing tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove unneeded comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: return nil when mtype doesn't match Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: update go fmt version Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: cleanup Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: document deepcopyparser Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: cover edgecase of `gauge_created` in CreatedTimestamp() Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: readability updates Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: dedeuplicate labeldiff checks Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add tests for new label functions Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: document CreatedTimestamp func Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: optimize `CreatedTimestamp()` - Use refactored CreatedTimestamp function with bug fixes - Remove unused code in labels.go - Improve code documentation Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: add tests and lint fixes Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove mName Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add tests for CT parse failures and deepCopy Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: edit expectCT struct Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: add new label in deepCopy Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: use p.builder in deepCopy Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: add NewMetricsParserWithOpts Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * chore: comments Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * chore: rename var Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: add condition for OM fuzzing Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: build tags Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: default skipCT to false Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: rename skipCT to skipCTSeries Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: formatting Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments and readability updates Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * refac: remove NewOpenMetricsParserWithOpts Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: extract skipCTSeries logic from parseMetricSuffix Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: inline create a NewOpenMetricsParser Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * refac: improve error handling Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: return error instead of nil Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: remove skipCT check from tBraceOpen Signed-off-by: Manik Rana <manikrana54@gmail.com> * Pair programming with Manik, Arthur and Daniel. Signed-off-by: bwplotka <bwplotka@gmail.com> * chore: comments and use helper funcs Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	2024-08-08 12:35:35 +01:00
Charles Korn	f91009aa2e	promql: clarify error message when panic occurs during query evaluation ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-08 09:11:38 +10:00
Björn Rabenstein	27579c9148	Merge pull request #14605 from krajorama/fix-staleness-pool-corrupt ... Fix histogram pool poisoning bug chunkenc.Iterator	2024-08-07 21:02:08 +02:00
George Krajcsovits	17b0b788da	Update promql/engine.go ... Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>	2024-08-07 20:15:46 +02:00
Charles Korn	0f4bc87b4f	Make linter happy ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 15:35:06 +10:00
Charles Korn	f07b3ae67b	Fix issue where avg over mixed exponential and custom buckets, or incompatible custom buckets, produces incorrect results or panics ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 15:32:35 +10:00
Charles Korn	5ee94f49a2	Fix issue where sum over mixed exponential and custom buckets, or incompatible custom buckets, produces incorrect results ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 15:30:01 +10:00
Charles Korn	424cefcf5e	Fix "cannot reduce resolution to custom buckets schema" panic in rate over native histograms with mix of custom and exponential buckets ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 14:45:02 +10:00
Charles Korn	f6e4b775e2	Check for errors first ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 14:25:46 +10:00
György Krajcsovits	b91acc61b0	Add unit test ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-06 20:50:51 +02:00
Björn Rabenstein	ee5bba07c0	Merge pull request #14413 from prometheus/beorn7/promql ... promql: more Kahan summation (avg) and less incremental mean calculation (avg, avg_over_time)	2024-08-06 19:56:32 +02:00
Björn Rabenstein	71ba554293	Merge pull request #14482 from ywwg/owilliams/group-quote ... promql: support quoting in grouping label lists	2024-08-06 19:43:38 +02:00
György Krajcsovits	37c8c9257b	Fix histogram pool poisoning bu chunkenc.Iterator ... chunkenc.Iterator.AtFloatHistogram may do a shallow copy if it receives nil as input pointer. This can in turn share the span slice with multiple histograms in the matrixSelectorHPool, leading to unexpected errors. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-06 19:40:14 +02:00
Charles Korn	aadec25faf	promql: Fix issue where some native histogram-related annotations are not emitted by rate (#14575) ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-06 09:10:40 +01:00
Owen Williams	d90c5a71d7	support quoting in grouping label lists ... Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-01 10:07:12 -04:00
Filip Petkovski	3e6155e596	Use CopyTo when resetting histogram in stats iterator ... The histogram stats iterator does not fully clear the histogram object and is not resilient to new fields being added to the histogram type. To resolve the issue, the commit uses the CopyTo methods which should be future proof to new fields being added. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2024-07-31 11:18:47 +02:00
Charles Korn	7fab72a280	promqltest: add support for setting counter reset hint on histogram samples (#14537) ... * promqltest: add support for setting counter reset hint on histogram samples Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-07-31 09:53:05 +02:00
Max Amin	84b819a69f	feat: add Google cloud roundtripper for remote write (#14346) ... * feat: Google Auth for remote write Signed-off-by: Max Amin <maxamin@google.com> --------- Signed-off-by: Max Amin <maxamin@google.com>	2024-07-30 16:25:19 +01:00
Filip Petkovski	2cd97c61e0	Add more test cases ... Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2024-07-29 14:53:32 +02:00
Filip Petkovski	be7a4c9b83	Ignore stale histograms for counter reset detection ... The histogram stats decoder keeps track of the last seen histogram sample in order to properly detect counter resets. We are seeing an issue where a histogram with UnknownResetHint gets treated as a counter reset when it follows a stale histogram sample. I believe that this is incorrect since stale samples should be completely ignored in PromQL. As a result, they should not be stored in the histogram stats iterator and the counter reset detection needs to be done against the last non-stale sample. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2024-07-26 10:08:31 +02:00
Björn Rabenstein	ca7062cf49	Merge pull request #14464 from prometheus/beorn7/histogram ... promql: Add NHCB tests	2024-07-17 14:57:46 +02:00
beorn7	c39776c5b5	promql: Add NHCB tests ... This adds equivalent NHCB tests to the existing classic histogram tests. Signed-off-by: beorn7 <beorn@grafana.com>	2024-07-16 12:20:43 +02:00
Bryan Boreham	bb241d7960	Merge pull request #14463 from fpetkovski/customize-acceptance-tests ... Allow built-in tests to be customized	2024-07-15 10:04:41 +01:00
Bryan Boreham	d116bf7b9f	Merge pull request #14109 from harry671003/pass_limit_to_querier ... storage: pass limit param as hint in querier	2024-07-12 10:27:52 +01:00