1183 commits

Author	SHA1	Message	Date
Arve Knudsen	f7b396a1dc	promql.Engine: Refactor vector selector evaluation into a method (#14900) ... New method is named `evalVectorSelector`. --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-10-15 14:57:54 +01:00
Neeraj Gartia	d4b1f9eb33	Corrects the behaviour of binary opperators between histogram and float (#14726) ... promql: corrects binary operators functioning for mixed sample with histogram and float For invalid pairings of sample types, an annotation is added now. Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-10-15 14:44:36 +02:00
George Krajcsovits	522149a2ae	model: move classic NHCB conversion into its own file (#15156) ... * model: move classic to NHCB conversion into its own file In preparation for #14978. Author: Jeanette Tan <jeanette.tan@grafana.com> 2024-07-03 11:56:48 Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Better naming from review comment Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Add doc strings. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-10-14 14:23:11 +02:00
Fiona Liao	8650d25804	Add additional basic nhcb unit tests (#15086) ... * Add additional basic nhcb unit tests * Update promql/promqltest/testdata/histograms.test Signed-off-by: Fiona Liao <fiona.liao@grafana.com> Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2024-10-08 14:34:32 +02:00
TJ Hoplock	6ebfbd2d54	chore!: adopt log/slog, remove go-kit/log ... For: #14355 This commit updates Prometheus to adopt stdlib's log/slog package in favor of go-kit/log. As part of converting to use slog, several other related changes are required to get prometheus working, including: - removed unused logging util func `RateLimit()` - forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger - move some of the json file logging functionality to use prom/common package functionality - refactored some of the new json file logging for scraping - changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers - updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition - added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2024-10-07 15:58:50 -04:00
Matthieu MOREL	ab64966e9d	fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094) ... * fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" --------- Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-10-06 16:35:29 +00:00
Björn Rabenstein	199d2fb492	Merge pull request #14989 from NeerajGartia21/eval_info ... Adds eval_info command to PromQL testing framework	2024-10-02 16:53:24 +02:00
Björn Rabenstein	78f792135c	Merge pull request #15025 from prometheus/notreset-bug ... Fix bug in rate vs float and histogram mixup	2024-10-02 14:17:55 +02:00
György Krajcsovits	b5479831b8	Unit test for regression in rate vs float and histogram mixup ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-02 13:56:33 +02:00
Manik Rana	f1c57a95ed	change: No longer ingest OM _created as timeseries if feature-flag 'enable-ct-zero-ingestion' is enabled; fixed OM text CT conversion bug (#14738) ... * chore: revert TypeRequiresCT to private Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: init NewOpenMetricsParser with skipCT true Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: allow opt-in to OM CT ingestion Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: use textparse interface to set om options Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: set skipOMSeries in test Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * wip: add tests for OM CR parse Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: merge ct tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add cases for OM text Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: check correct test cases Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: use both scrape protocols in config Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: fix inputs and output tests for OM Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: cleanup Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: rename skipOMSeries to skipOMCTSeries Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * fix: finish refac Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: move setup code outside test Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: verify _created lines create new metric in certain cases Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: post merge fixes Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * manager: Fixed CT OMText conversion bug; Refactored tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: imports Signed-off-by: Manik Rana <manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	2024-10-02 11:52:03 +01:00
Bryan Boreham	b8e5b7cda9	[REFACTOR] PromQL: remove label_join and label_replace stubs ... These functions operate on whole series, not on samples, so they do not fit into the table of functions that return a Vector. Remove the stub entries that were left to help downstream users of the code identify what changed. We cannot remove the entries from the `FunctionCalls` map without breaking `TestFunctionList`, so put some nils in to keep it happy. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-09-27 11:20:45 +01:00
Neeraj Gartia	8aef821e10	eval_info command ... Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-09-27 01:10:00 +05:30
Bryan Boreham	7f99d2930d	[BUGFIX] PromQL: make sort_by_label stable ... Go's sorting functions can re-order equal elements, so the strategy of sorting by the fallback ordering first does not always work. Pulling the fallback into the main comparison function is more reliable and more efficient. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-09-26 11:12:33 +01:00
Arve Knudsen	c2bbabb4a7	promql.Engine: Refactor vector selector evaluation into a method (#14900) ... * PromQL.Engine: Refactor Matrix expansion into a method Add utility method promql.evaluator.expandSeriesToMatrix, for expanding a slice of storage.Series into a promql.Matrix. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Rename to generateMatrix Rename evaluator.expandSeriesToMatrix into generateMatrix, while also dropping the start, end, interval arguments since they are evaluator fields. Write more extensive method documentation. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Rename to evalVectorSelector Rename to evalVectorSelector after discussing with @michahoffmann. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-09-24 11:03:56 +01:00
Arthur Silva Sens	6bcb064d93	Merge pull request #14950 from Maniktherana/fuzz-om-minor-change ... chore: remove unused code	2024-09-21 09:22:17 +01:00
Manik Rana	3fba6bcdfa	chore: remove unused code ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-09-20 22:00:15 +05:30
Björn Rabenstein	5b9148e552	Merge pull request #14820 from charleskorn/promqltest-native-histogram-format ... promqltest: use test expression format for histograms in assertion failure messages and include reset hint in the test expression	2024-09-20 16:47:08 +02:00
Björn Rabenstein	df9916ef66	Merge pull request #14677 from prometheus/beorn7/histogram ... promql(native histograms): Introduce exponential interpolation	2024-09-19 18:08:59 +02:00
Jan Fajerski	96e5a94d29	promql: rename holt_winters to double_exponential_smoothing ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-09-19 15:29:01 +02:00
Björn Rabenstein	1639450172	Merge pull request #14821 from charleskorn/nh-negative-multiplication-division ... promql: correctly handle unary negation of native histograms and add tests for multiplication and division of native histograms by negative scalars Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-09-19 14:07:37 +01:00
beorn7	6fcd225aee	promql(native histograms): Introduce exponential interpolation ... The linear interpolation (assuming that observations are uniformly distributed within a bucket) is a solid and simple assumption in lack of any other information. However, the exponential bucketing used by standard schemas of native histograms has been chosen to cover the whole range of observations in a way that bucket populations are spread out over buckets in a reasonably way for typical distributions encountered in real-world scenarios. This is the origin of the idea implemented here: If we divide a given bucket into two (or more) smaller exponential buckets, we "most naturally" expect that the samples in the original buckets will split among those smaller buckets in a more or less uniform fashion. With this assumption, we end up with an "exponential interpolation", which therefore appears to be a better match for histograms with exponential bucketing. This commit leaves the linear interpolation in place for NHCB, but changes the interpolation for exponential native histograms to exponential. This affects `histogram_quantile` and `histogram_fraction` (because the latter is more or less the inverse of the former). The zero bucket has to be treated specially because the assumption above would lead to an "interpolation to zero" (the bucket density approaches infinity around zero, and with the postulated uniform usage of buckets, we would end up with an estimate of zero for all quantiles ending up in the zero bucket). We simply fall back to linear interpolation within the zero bucket. At the same time, this commit makes the call to stick with the assumption that the zero bucket only contains positive observations for native histograms without negative buckets (and vice versa). (This is an assumption relevant for interpolation. It is a mostly academic point, as the zero bucket is supposed to be very small anyway. However, in cases where it _is_ relevantly broad, the assumption helps a lot in practice.) This commit also updates and completes the documentation to match both details about interpolation. As a more high level note: The approach here attempts to strike a balance between a more simplistic approach without any assumption, and a more involved approach with more sophisticated assumptions. I will shortly describe both for reference: The "zero assumption" approach would be to not interpolate at all, but _always_ return the harmonic mean of the bucket boundaries of the bucket the quantile ends up in. This has the advantage of minimizing the maximum possible relative error of the quantile estimation. (Depending on the exact definition of the relative error of an estimation, there is also an argument to return the arithmetic mean of the bucket boundaries.) While limiting the maximum possible relative error is a good property, this approach would throw away the information if a quantile is closer to the upper or lower end of the population within a bucket. This can be valuable trending information in a dashboard. With any kind of interpolation, the maximum possible error of a quantile estimation increases to the full width of a bucket (i.e. it more than doubles for the harmonic mean approach, and precisely doubles for the arithmetic mean approach). However, in return the _expectation value_ of the error decreases. The increase of the theoretical maximum only has practical relevance for pathologic distributions. For example, if there are thousand observations within a bucket, they could _all_ be at the upper bound of the bucket. If the quantile calculation picks the 1st observation in the bucket as the relevant one, an interpolation will yield a value close to the lower bucket boundary, while the true quantile value is close to the upper boundary. The "fancy interpolation" approach would be one that analyses the _actual_ distribution of samples in the histogram. A lot of statistics could be applied based on the information we have available in the histogram. This would include the population of neighboring (or even all) buckets in the histogram. In general, the resolution of a native histogram should be quite high, and therefore, those "fancy" approaches would increase the computational cost quite a bit with very little practical benefits (i.e. just tiny corrections of the estimated quantile value). The results are also much harder to reason with. Signed-off-by: beorn7 <beorn@grafana.com>	2024-09-19 14:19:10 +02:00
Jan Fajerski	15cea39136	promql: put holt_winters behind experimental feature flag ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-09-18 15:39:58 +02:00
Joshua Hesketh	b6107cc888	Make rate possible non-counter annotation consistent (#14910) ... * Make rate possible non-counter annotation consistent Previously a PossibleNonCounterInfo annotation would be left in cases where a range-vector selects 1 float data point, even if no more points are selected in order to calculate a rate. This change ensures an output float exists before emitting such an annotation. This fixes an inconsistency where a series with mixed data (ie, a float and a native histogram) would emit an annotation without any points. For example, ``` load 1m series{label="a"} 1 {{schema:1 sum:10 count:5 buckets:[1 2 3]}} eval instant at 1m rate(series[1m1s]) ``` Would have a PossibleNonCounterInfo annotation. Wheras ``` load 1m series{label="a"} {{schema:1 sum:10 count:5 buckets:[1 2 3]}} {{schema:1 sum:15 count:10 buckets:[1 2 3]}} eval instant at 1m rate(series[1m1s]) ``` Would not. --------- Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	2024-09-18 10:21:25 +00:00
Mario Fernandez	5814920601	Fix: optimize .* regexp performance ... Shortcut for `.*` matches newlines as well. Add preamble change ^(?s: Add test dotAll flag por al regex Add and fix regex tests Signed-off-by: Mario Fernandez <mariofer@redhat.com>	2024-09-17 12:18:31 +02:00
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868) ... * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	2024-09-10 22:32:03 +02:00
Jan Fajerski	91608c002f	Merge branch 'main' into release-3.0-beta.0 ... Conflicts: scrape/scrape_test.go Pick both changes.	2024-09-10 20:51:20 +02:00
Björn Rabenstein	8114f9a281	Merge pull request #14821 from charleskorn/nh-negative-multiplication-division ... promql: correctly handle unary negation of native histograms and add tests for multiplication and division of native histograms by negative scalars	2024-09-10 15:23:00 +02:00
Jan Fajerski	fa318711f4	Merge branch 'main' into 3.0-main-sync-24-09-09 ... Conflicts: cmd/prometheus/main.go docs/command-line/prometheus.md docs/feature_flags.md web/ui/build_ui.sh web/web.go Resolved by dropping the UTF-8 feature flag and adding the `auto-reload-config` feature flag. For the new web ui pick all changes from `main`.	2024-09-09 15:44:22 +02:00
Charles Korn	e8c7482137	Return negative counts when multiplied or divided by a negative value ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-09 14:37:59 +10:00
Charles Korn	2bdb3452d1	Modify parser for native histograms to allow negative values where supported ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-09 14:37:58 +10:00
Charles Korn	9852855084	Implement unary negation for native histograms ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-09 14:37:55 +10:00
Arve Knudsen	db5e48dc33	promql.Engine.Close: No-op if nil (#14861) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-09-08 14:39:13 +02:00
Julien Pivotto	ec73d8dd29	promqltest: run each eval command in a subtest ... Simpler approach to #14576 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-09-06 13:05:46 +02:00
Björn Rabenstein	d2b659540e	Merge pull request #14819 from prometheus/beorn7/doc ... promql: Improve the rate extrapolation explanation	2024-09-05 12:21:03 +02:00
Jan Fajerski	befcfadf78	Fix merge conflicts ... Fix call to newTestEngine(t) in promql/engine_test.go:3214. `agent` feature-flag it's own cmdline flag now. Remove `scrape.name-escaping-scheme` argument. Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-09-04 20:06:17 +02:00
Jan Fajerski	fe4289b502	Merge branch 'main' into HEAD	2024-09-04 18:50:00 +02:00
Bryan Boreham	485523eed2	Merge pull request #14816 from bboreham/improve-promql-tracing ... Improve promql tracing	2024-09-04 14:32:22 +01:00
Arve Knudsen	6ef1eb4e68	Merge pull request #14064 from aknuds1/arve/close-engine ... Release PromQL engine resources on close, from tests	2024-09-04 12:07:35 +02:00
Charles Korn	113de6301c	Add failing test cases for unary negation and multiplication and division with negative scalars ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-04 16:20:28 +10:00
Charles Korn	9b451abec7	Make positive and negative bucket counts different in existing test cases ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-04 16:08:05 +10:00
Charles Korn	90dc1b45db	promqltest: use test expression format for histograms in assertion failure messages ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-04 15:47:07 +10:00
Charles Korn	e67358d203	histogram: include counter reset hint in test expression output ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-09-04 15:46:52 +10:00
beorn7	2304be9dd5	promql: Improve the rate extrapolation explanation ... Signed-off-by: beorn7 <beorn@grafana.com>	2024-09-03 22:11:14 +02:00
Bryan Boreham	abb0502685	[ENHANCEMENT] PromQL: Add detail to tracing spans ... For aggregates, operators, calls, show what operation is performed. Also add an event when series are expanded, typically time spent accessing TSDB. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-09-03 14:15:58 +01:00
Bryan Boreham	54989ce6fd	PromQL: add short string method ... To be used in tracing or logging. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-09-03 14:15:58 +01:00
Bryan Boreham	8742077498	[BUGFIX] PromQL: pass Context so spans parent correctly ... Assigning to `evaluator.ctx` in `eval()` broke the parent-child relationship. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-09-03 14:15:29 +01:00
Arve Knudsen	70bb219d33	Merge pull request #14680 from charleskorn/promqltest-scalar ... promqltest: improve error messages when unexpected type is returned, and fix inverted expected and actual values in error message	2024-09-03 10:38:04 +02:00
Arve Knudsen	5dfbcc390e	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-09-02 16:26:59 +02:00
Jan Fajerski	956245b25b	promqltest: adjust eval times and range selector ... In order to fix new tests for changes added in https://github.com/prometheus/prometheus/pull/13904. Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-09-02 11:27:39 +02:00
Jan Fajerski	00315ce15e	Merge branch 'main' into 3.0-main-sync-24-08-30 ... using -Xours Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-09-02 11:27:18 +02:00
Neeraj Gartia	8c7bf39d96	Moves TestNativeHistogram_MulDivOperator to promql testing framework (#14688) ... PromQL: add test for mul and div operator Also, remove the converted test from the engine_test.go file. This also includes an extension of the test framework to allow NaN/Inf in histogram buckets. --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-08-29 16:42:35 +02:00
Jorge Creixell	e9e3d64b7c	PromQL engine: Delay deletion of __name__ label to the end of the query evaluation (#14477) ... PromQL engine: Delay deletion of __name__ label to the end of the query evaluation - This change allows optionally preserving the `__name__` label via the `label_replace` and `label_join` functions, and helps prevent the dreaded "vector cannot contain metrics with the same labelset" error. - The implementation extends the `Series` and `Sample` structs with a boolean flag indicating whether the `__name__` label should be deleted at the end of the query evaluation. - The `label_replace` and `label_join` functions can still access the value of the `__name__` label, even if it has been previously marked for deletion. If `__name__` is used as target label, it won't be dropped at the end of the query evaluation. - Fixes https://github.com/prometheus/prometheus/issues/11397 - See https://github.com/jcreixell/prometheus/pull/2 for previous discussion, including the decision to create this PR and benchmark it before considering other alternatives (like refactoring `labels.Labels`). - See https://github.com/jcreixell/prometheus/pull/1 for an alternative implementation using a special label instead of boolean flags. - Note: a feature flag `promql-delayed-name-removal` has been added as it changes the behavior of some "weird" queries (see https://github.com/prometheus/prometheus/issues/11397#issuecomment-1451998792) Example (this always fails, as `__name__` is being dropped by `count_over_time`): ``` count_over_time({__name__!=""}[1m]) => Error executing query: vector cannot contain metrics with the same labelset ``` Before: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => Error executing query: vector cannot contain metrics with the same labelset ``` After: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => count_go_gc_cycles_automatic_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 count_go_gc_cycles_forced_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 ... ``` Signed-off-by: Jorge Creixell <jcreixell@gmail.com> --------- Signed-off-by: Jorge Creixell <jcreixell@gmail.com> Signed-off-by: Björn Rabenstein <github@rabenste.in>	2024-08-29 15:50:39 +02:00
Arve Knudsen	99204f23ee	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-29 09:52:54 +02:00
Björn Rabenstein	849215d90c	Merge pull request #14585 from fatsheep9146/covert-TestNativeHistogram_Sum_Count_Add_AvgOperator-to-framework ... convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework	2024-08-28 17:21:32 +02:00
Owen Williams	9da75328ea	fix(utf8): ensure correct validation when legacy mode turned on (#14736) ... fix(utf8): ensure correct validation when legacy mode turned on This depends on the included update of the prometheus/common dependency. --------- Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-28 17:15:42 +02:00
Arve Knudsen	c9a460d570	Merge remote-tracking branch 'prometheus/main' into arve/close-engine ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-26 12:17:10 +02:00
Ziqi Zhao	87eab0aaad	fix golang lint ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-23 09:32:04 +08:00
Ziqi Zhao	c7c4a5c347	add helper function to compare native histograms in testing ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-23 09:28:31 +08:00
beorn7	0f760f63dd	lint: Revamp our linting rules, mostly around doc comments ... Several things done here: - Set `max-issues-per-linter` to 0 so that we actually see all linter warnings and not just 50 per linter. (As we also set `max-same-issues` to 0, I assume this was the intention from the beginning.) - Stop using the golangci-lint default excludes (by setting `exclude-use-default: false`. Those are too generous and don't match our style conventions. (I have re-added some of the excludes explicitly in this commit. See below.) - Re-add the `errcheck` exclusion we have used so far via the defaults. - Exclude the signature requirement `govet` has for `Seek` methods because we use non-standard `Seek` methods a lot. (But we keep other requirements, while the default excludes completely disabled the check for common method segnatures.) - Exclude warnings about missing doc comments on exported symbols. (We used to be pretty adamant about doc comments, but stopped that at some point in the past. By now, we have about 500 missing doc comments. We may consider reintroducing this check, but that's outside of the scope of this commit. The default excludes of golangci-lint essentially ignore doc comments completely.) - By stop using the default excludes, we now get warnings back on malformed doc comments. That's the most impactful change in this commit. It does not enforce doc comments (again), but _if_ there is a doc comment, it has to have the recommended form. (Most of the changes in this commit are fixing this form.) - Improve wording/spelling of some comments in .golangci.yml, and remove an outdated comment. - Leave `package-comments` inactive, but add a TODO asking if we should change that. - Add a new sub-linter `comment-spacings` (and fix corresponding comments), which avoids missing spaces after the leading `//`. Signed-off-by: beorn7 <beorn@grafana.com>	2024-08-22 17:36:11 +02:00
Jan Fajerski	7c8c748399	promql tests: adjust range query intervals ... Some test queries need their interval adjusted to account for https://github.com/prometheus/prometheus/pull/13904. Otherwise the queries don't return enough samples. promql/engine_test.go:TestHistogramCopyFromIteratorRegression needed the same, but also the result needed a fix since `increase` interpolates over the full range. Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-08-21 12:33:52 +02:00
Björn Rabenstein	7fad1ec8ee	Merge pull request #14655 from suntala/suntala/sort-by-label-enhancement ... promql: Fall back to full label sets when sorting by label	2024-08-21 12:28:55 +02:00
Jan Fajerski	5138922b0d	Merge branch 'main' into 3.0-main-sync-24-08-21	2024-08-21 09:09:36 +02:00
Ziqi Zhao	8f828d45c1	convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-21 09:24:50 +08:00
suntala	8e82ac8d2b	Add comments to the sort by label functions ... Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-20 20:41:55 +02:00
Arve Knudsen	3a78e76282	Upgrade golangci-lint to v1.60.1 ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-18 12:13:25 +02:00
Charles Korn	6548421f61	promqltest: improve error messages when unexpected type is returned, and fix inverted expected and actual values in error message when scalar value does not match expected ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-16 13:37:05 +10:00
Björn Rabenstein	1daf7cdd62	Merge pull request #14626 from cuiweiyuan/main ... chore: fix some function names	2024-08-15 11:46:21 +02:00
cuiweiyuan	1800af54f0	chore: fix some function names ... Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>	2024-08-15 13:57:21 +08:00
Charles Korn	52818a97e2	Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms ... # Conflicts: # promql/promqltest/testdata/native_histograms.test	2024-08-14 07:52:08 +10:00
Björn Rabenstein	c2bc6cfe97	Merge pull request #14621 from charleskorn/panic-message ... promql: clarify error message logged when panic occurs during query evaluation	2024-08-13 23:02:43 +02:00
George Krajcsovits	fe5354df88	Merge pull request #14658 from krajorama/zero-rate-nh-quantile ... Native histograms: define behavior when rate is null.	2024-08-13 16:05:43 +02:00
György Krajcsovits	386fc8b9f6	Update from review comments. ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-13 15:26:07 +02:00
Arve Knudsen	0503d4f372	PromQL: Fix comment regarding non-nil histogram pointer ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-13 08:55:24 +02:00
György Krajcsovits	6aee5b4b38	fix typo ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-12 12:04:45 +02:00
György Krajcsovits	06a8886b94	Native histograms: define behavior when rate is null. ... Histogram quantile returns NaN in this case, which might be surprising, so add a unit test that clarifies that this is intentional. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-12 10:40:21 +02:00
suntala	fd2f44af7f	Fall back to comparing by label set when sorting by label desc ... Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com> Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-11 21:44:03 +02:00
suntala	94ad489328	Fall back to comparing by label set when sorting by label ... Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com> Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-11 21:44:03 +02:00
Charles Korn	f992f81bd0	Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms ... Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com>	2024-08-09 13:58:54 +10:00
Charles Korn	5cfdde327c	Address PR feedback: add extra test case ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-09 13:57:37 +10:00
Charles Korn	82bb35fabb	Address PR feedback: fix typo and rename variable ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-09 13:51:31 +10:00
Björn Rabenstein	41f8a0223c	Merge pull request #14609 from charleskorn/rate-over-mixed-custom-exponential-histograms ... promql: fix "cannot reduce resolution to custom buckets schema" panic in `rate` over native histograms with mix of custom and exponential buckets	2024-08-08 14:03:41 +02:00
Manik Rana	02c465bf58	textparse: Implement CreatedTimestamp() in openmetricsparse.go (#14356) ... * feat: initial implement of createedTimestamp() with tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: return ct after finding it Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove unneeded test Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: add comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: multiple changes - implement changes from pair programming session - use newParse.val() - advance parser p if ct is found Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: check if err from p.Next() Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: advance parser and parse histograms + summary Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: restore previous tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: retore failing tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove unneeded comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: return nil when mtype doesn't match Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: update go fmt version Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: cleanup Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: document deepcopyparser Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: cover edgecase of `gauge_created` in CreatedTimestamp() Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: readability updates Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: dedeuplicate labeldiff checks Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add tests for new label functions Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: document CreatedTimestamp func Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: optimize `CreatedTimestamp()` - Use refactored CreatedTimestamp function with bug fixes - Remove unused code in labels.go - Improve code documentation Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: add tests and lint fixes Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: remove mName Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add tests for CT parse failures and deepCopy Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: edit expectCT struct Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: add new label in deepCopy Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: use p.builder in deepCopy Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: add NewMetricsParserWithOpts Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * chore: comments Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * chore: rename var Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: add condition for OM fuzzing Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: build tags Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: default skipCT to false Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: rename skipCT to skipCTSeries Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: formatting Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments and readability updates Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * refac: remove NewOpenMetricsParserWithOpts Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: extract skipCTSeries logic from parseMetricSuffix Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: inline create a NewOpenMetricsParser Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: comments Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * refac: improve error handling Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: return error instead of nil Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: remove skipCT check from tBraceOpen Signed-off-by: Manik Rana <manikrana54@gmail.com> * Pair programming with Manik, Arthur and Daniel. Signed-off-by: bwplotka <bwplotka@gmail.com> * chore: comments and use helper funcs Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	2024-08-08 12:35:35 +01:00
Charles Korn	f91009aa2e	promql: clarify error message when panic occurs during query evaluation ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-08 09:11:38 +10:00
Björn Rabenstein	27579c9148	Merge pull request #14605 from krajorama/fix-staleness-pool-corrupt ... Fix histogram pool poisoning bug chunkenc.Iterator	2024-08-07 21:02:08 +02:00
George Krajcsovits	17b0b788da	Update promql/engine.go ... Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>	2024-08-07 20:15:46 +02:00
Charles Korn	0f4bc87b4f	Make linter happy ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 15:35:06 +10:00
Charles Korn	f07b3ae67b	Fix issue where avg over mixed exponential and custom buckets, or incompatible custom buckets, produces incorrect results or panics ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 15:32:35 +10:00
Charles Korn	5ee94f49a2	Fix issue where sum over mixed exponential and custom buckets, or incompatible custom buckets, produces incorrect results ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 15:30:01 +10:00
Charles Korn	424cefcf5e	Fix "cannot reduce resolution to custom buckets schema" panic in rate over native histograms with mix of custom and exponential buckets ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 14:45:02 +10:00
Charles Korn	f6e4b775e2	Check for errors first ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-07 14:25:46 +10:00
György Krajcsovits	b91acc61b0	Add unit test ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-06 20:50:51 +02:00
Björn Rabenstein	ee5bba07c0	Merge pull request #14413 from prometheus/beorn7/promql ... promql: more Kahan summation (avg) and less incremental mean calculation (avg, avg_over_time)	2024-08-06 19:56:32 +02:00
Björn Rabenstein	71ba554293	Merge pull request #14482 from ywwg/owilliams/group-quote ... promql: support quoting in grouping label lists	2024-08-06 19:43:38 +02:00
György Krajcsovits	37c8c9257b	Fix histogram pool poisoning bu chunkenc.Iterator ... chunkenc.Iterator.AtFloatHistogram may do a shallow copy if it receives nil as input pointer. This can in turn share the span slice with multiple histograms in the matrixSelectorHPool, leading to unexpected errors. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-06 19:40:14 +02:00
Charles Korn	aadec25faf	promql: Fix issue where some native histogram-related annotations are not emitted by rate (#14575) ... Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-08-06 09:10:40 +01:00
Owen Williams	d90c5a71d7	support quoting in grouping label lists ... Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-01 10:07:12 -04:00
Jan Fajerski	8a96a75ba8	Merge branch 'main' into 3.0-main-sync-24-08-01 ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-08-01 15:44:47 +02:00
Filip Petkovski	3e6155e596	Use CopyTo when resetting histogram in stats iterator ... The histogram stats iterator does not fully clear the histogram object and is not resilient to new fields being added to the histogram type. To resolve the issue, the commit uses the CopyTo methods which should be future proof to new fields being added. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>	2024-07-31 11:18:47 +02:00
Charles Korn	7fab72a280	promqltest: add support for setting counter reset hint on histogram samples (#14537) ... * promqltest: add support for setting counter reset hint on histogram samples Signed-off-by: Charles Korn <charles.korn@grafana.com>	2024-07-31 09:53:05 +02:00
Max Amin	84b819a69f	feat: add Google cloud roundtripper for remote write (#14346) ... * feat: Google Auth for remote write Signed-off-by: Max Amin <maxamin@google.com> --------- Signed-off-by: Max Amin <maxamin@google.com>	2024-07-30 16:25:19 +01:00