Commit graph

953 commits

Author SHA1 Message Date
Björn Rabenstein e6be4240be
Merge pull request #14068 from colega/quote-label-name-in-matchers-when-needed
Bugfix: quote label name in matchers when needed
2024-05-14 17:18:58 +02:00
Bryan Boreham 3fd24d1cd7
Merge pull request #13999 from bboreham/extract-promqltest
[Test] Extract most PromQL test code into separate packages
2024-05-09 13:23:11 +01:00
Bryan Boreham 786e0e706c test: PromQL: stop using internal fields of engine
* set enablePerStepStats and lookback duration via
  `NewTestEngine` parameters.
* check maxSamples by recreating query engine
* check lookback without modifying internals

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 19:58:41 +01:00
Bryan Boreham 0dbfd20b69 test: move most PromQL tests into separate test package
So that they can import promqltest which imports promql.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:28:56 +01:00
Bryan Boreham 4a72607c4a refactor: extract some PromQL Engine tests which use unexported structs
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:27:17 +01:00
Bryan Boreham e7c77f7b40 promql: export NewTestQuery
So that tests can call it from another package.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:08:04 +01:00
Bryan Boreham babfcfdd91 refactor: Move NewTestEngine into promqltest
And export `DefaultMaxSamplesPerQuery` so callers can replicate previous
behaviour.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:07:28 +01:00
Oleg Zaytsev dabd789fd5
Quote label name in matchers when needed
When the label name of a matcher contains non-standard characters, like
a dot, or starts with a digit, it should be quoted.

If it's not quoted, then `VectorSelector.String()` isn't a valid PromQL.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-05-08 16:58:51 +02:00
Bryan Boreham b3c15d2246 test: clean up promqltest package references
So it nearly compiles.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham 9aa321d672 test: make field initializers explicit
Lint started complaining after I moved the file.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham ccf73266f0 test: move promqltest tests together with the implementation
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham 11b27d5d22 test: move test files into new promqltest package
So that promql package does not bring in test-only dependencies.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham a6e427660a test: check for @-modifier without using engine internals
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham 2b0c87b1b6 test: turn TestKahanSum into scripted test
This saves having a function solely to call kahanSumInc.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham a1af3c27d4 refactor: extract almost.Equal() to new package
To avoid a circular reference between promql and promqltest.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:10 +01:00
Arve Knudsen a25160e6a4
[REFACTOR] PromQL: simplify rangeEvalTimestampFunctionOverVectorSelector (#14021)
The function `rangeEvalTimestampFunctionOverVectorSelector` appeared to be checking histogram size, however the value it used was always 0 due to subtle variable shadowing.
However we don't need to pass sample values to the `timestamp` function, since the latter only cares about timestamps. This also affects peak sample count in statistics, since we are no longer copying histogram samples.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-08 10:39:44 +01:00
Oleg Zaytsev 4b7a44c7a0
Fix parser.VectorSelector.String() with empty name matcher (#14015)
The check fell into "this matcher equals vector selector's name" case when vector selector doesn't have a name and the matcher is an explicit matcher for an empty __name__ label.

To provide some context about why this is important: some downstream projects use the promql.Parse(expr.String()) to clone an expression's AST, and with this bug that matcher disappears in the cloning.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-05-06 11:51:08 +02:00
guangwu 9fda9443d4
fix(promql/query_logger): close file in error handling (#13948)
Signed-off-by: guoguangwu <guoguangwug@gmail.com>
2024-04-30 10:47:10 +02:00
Owen Williams 4a6f8704ef
parser: remake generated_parser output (#13923)
In a previous PR, the generated parser was created using an old version of goyacc.

Also adds -l to disable line directives, which fixes debug processing and reduces diffs at the expense of making it more difficult to reason about the generated output.

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-04-13 12:59:54 +02:00
Neeraj Gartia 612de026da
Adds Inf and NaN as Numbers to Histogram in Promql Testing Framework (#13916)
includes Inf and NaN as numbers to histogram

---------

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
Signed-off-by: Björn Rabenstein <github@rabenste.in>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
2024-04-11 12:53:28 +02:00
Matthieu MOREL 6f595c6762
golangci-lint: enable whitespace linter (#13905)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-04-11 09:27:54 +01:00
Bryan Boreham 12961c6a37 promql: refactor: eliminate one 'else'
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 0ac927515b promql: move group-seen into group struct
Save allocating an auxilliary array.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 7499d90913 promql: remove pointer to aggregation groups
Just allocate in one slice.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham cfbeb6681b promql: re-use one heap for topk and bottomk
Slightly ugly casting saves memory.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 5e3914a27c promql: remove histogramMean from groupedAggregation
Re-use histogramValue since we don't need them separately.

Tidy up initialization.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 2cf3c9de8f promql: store labels per-group only for count_values
This saves memory in other kinds of aggregation.

We don't need `orderedResult` in `aggregationCountValues`; the ordering
is not guaranteed.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 185290a0d2 promql: pull checking of q and k out of loop
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 4584f67e17 promql: inline nextSample function
Move Sample out of loop to reduce allocations, otherwise it escapes to
the heap.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 526ce4ee7a promql: simplify data collection in aggregations
We don't need a Sample, just the float and histogram values.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 2f03acbafc promql: refactor: split topk/bottomk from sum/avg/etc
They aggregate results in different ways.
topk/bottomk don't consider histograms so can simplify data collection.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 74eed67ef6 promql: refactor: pull fetching input data out of rangeEvalAgg
This is a cleaner split of responsibilities.
We now check the sample count after calling rangeEvalAgg.
Changed re-use of samples to use `Clone` and `defer`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 602eb69edf promql: refactor: extract function nextSample
With sub-function nextValues which we shall use shortly.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham eb41e770b7 promql: refactor: extract function addToSeries
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 53a3138eeb promql aggregations: pre-generate mapping from inputs to outputs
So we don't have to re-create it on every time step.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham cb6c4b3092 promql: simplify k/q parameter to topk/bottomk/quantile
Pass it as a float64 not as interface{}.
Make k a simple int, since that is the parameter to make().
Pull invalid quantile warning out of the loop.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham b3bda7df4b promql: aggregations: skip copying input to a Vector
We can work directly from the inputMatrix on each timestep.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham c9b6c4c55a promql: aggregations: output directly to matrix for instant queries
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 3851b74db1 promql: aggregations: skip result vector in range queries
Adjust test to match the lower count, since samples in the vector
are no longer counted.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 59548b8a0b promql: refactor: move collection of results into aggregation()
We don't need to check for duplicates as aggregation cannot generate them.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham bd9bdccb22 promql: refactor: simplify internal data structures
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 5f10d17cef promql: refactor: split out aggregations over range
The new function `rangeEvalAgg` is mostly a copy of `rangeEval`, but
without `initSeries` which we don't need and inlining the callback to
`aggregation()`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham e5f667537c promql: refactor: initialize aggregation before storing in map
This seems more consistent to me.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 29244fb841 promql: refactor: extract count_values implementation
The existing aggregation function is very long and covers very different
cases.

`aggregationCountValues` is just for `count_values`, which differs from
other aggregations in that it outputs as many series per group as there
are values in the input.

Remove the top-level switch on string parameter type; use the same `Op`
check there as elswehere.

Pull checking parameters out to caller, where it is only executed once.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 8e04ab6dd4 promql: refactor: extract generateGroupingLabels function
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 7c28521451 [TESTS] Truncate some long test names, for readability
The strings produced by these tests can run to thousands of characters,
which makes test logs difficult to read.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-03 10:10:39 +01:00
Charles Korn cd72ebb05f
promql: include more details in error message when creating test query fails or an unexpected series is returned (#13847)
* promql: include more details in error message when creating test query fails

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Include more details when an unexpected metric is returned

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-04-03 10:57:08 +02:00
Julius Volz 9b7de47787
Remove unused Dmn field on EvalNodeHelper (#13877)
https://github.com/prometheus/prometheus/pull/13446 removed the last usage of
this field, but didn't remove the field.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2024-04-02 18:45:46 +02:00
beorn7 65b4696b88 promql: Remove leftover debug output
Signed-off-by: beorn7 <beorn@grafana.com>
2024-03-27 19:02:27 +01:00
beorn7 2c1f9558b2 promql: Fix histogram comparison in test framework
The definition of histograms in the test framework may create
histograms in a non-compact form. Since histogram comparison relies on
exact equality of the bucket layout, we have to compact the histograms
created by the test framework language before comparing them to
histograms returned from the PromQL engine.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-03-27 19:00:16 +01:00