Commit graph

979 commits

Author SHA1 Message Date
Bryan Boreham b6aba4ff14
Merge pull request #14074 from bboreham/kahan-sum-sum
[ENHANCEMENT] PromQL: use Kahan summation for sum()
2024-06-24 11:13:26 +01:00
Bryan Boreham 84602bbace
Merge branch 'main' into fix-matcher-string-with-empty-label-name 2024-06-19 05:56:25 -04:00
Charles Korn aeec30f082
Convert TestTimestampFunction_StepsMoreOftenThanSamples
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-17 16:56:56 +10:00
Charles Korn 987fa5c6a2
Convert range query test cases to test scripting language
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-17 16:43:01 +10:00
Oleg Zaytsev 03cf6141d4
Fix Matcher.String() with empty label name
When the label name is empty, which can happen now with quoted label
name, it should be quoted when printed as a string again.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-06-13 18:46:35 +02:00
Björn Rabenstein 08621bebe9
Merge pull request #14269 from prometheus/beorn7/histogram-test
promql: Add tests for histogram counter reset only in bucket
2024-06-08 16:59:47 +02:00
Charles Korn 1f988f77ff
promql: extend test scripting language to support asserting on expected error message (#14038)
Add ability to assert that a query fails with a particular error message

This also adds documentation for the test scripting language in general,
including the new feature.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-06 17:56:25 +02:00
beorn7 c7fdfe8004 promql: Add tests for histogram counter reset only in bucket
This also exercises the "fast path" (only decoding count and sum),
i.e. where the counter reset isn't visible at all in the decoded data.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-06-06 17:47:38 +02:00
Filip Petkovski 6e68046c25
Implement histogram statistics decoder (#14097)
Implement histogram statistics decoder

This commit speeds up histogram_count and histogram_sum
functions on native histograms. The idea is to have separate decoders which can be
used by the engine to only read count/sum values from histogram objects. This should help
with reducing allocations when decoding histograms, as well as with speeding up aggregations
like sum since they will be done on floats and not on histogram objects.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

---------

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
2024-06-06 17:17:13 +02:00
Charles Korn 24ef000025
Merge branch 'main' into sort-matrix-series
# Conflicts:
#	promql/engine_test.go
2024-05-31 12:44:27 +10:00
Arve Knudsen f3b8750339 Join errors
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-27 17:14:17 +02:00
Arve Knudsen 7b56353090 Merge remote-tracking branch 'prometheus/main' into arve/query-logger-munmap 2024-05-27 17:08:33 +02:00
Björn Rabenstein 114dc5c393
Merge pull request #13638 from NeerajGartia21/promql-test
Converts existing native histogram unit tests to the PromQL testing framework
2024-05-19 19:35:36 +02:00
Arve Knudsen 0f01d4b336 Fix flaky test
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-15 21:58:56 +02:00
Arve Knudsen bf8d88f326 Merge remote-tracking branch 'origin/main' into arve/query-logger-munmap 2024-05-15 21:02:03 +02:00
Oleksandr Redko f10c3454e9 Enable perfsprint linter and fix up code
Signed-off-by: Oleksandr Redko <oleksandr.red+github@gmail.com>
2024-05-15 17:51:05 +03:00
Björn Rabenstein e6be4240be
Merge pull request #14068 from colega/quote-label-name-in-matchers-when-needed
Bugfix: quote label name in matchers when needed
2024-05-14 17:18:58 +02:00
Neeraj Gartia 661856cb65 removes the added tests from engine_test.go
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:58:25 +05:30
Neeraj Gartia 6119124d0e some nits
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:55:38 +05:30
Neeraj Gartia adf5a36c1e adds test for sum, count, stddev, stdvar, quantile and fraction func to promql testing framework
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:55:38 +05:30
Neeraj Gartia 8b838a05d9 adds test for native histogram rate func in promql testing framework
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:55:38 +05:30
Neeraj Gartia 548bd9d6fb adds TestNativeHistogramRate func to promql test framework
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:55:38 +05:30
Charles Korn 0e934dba8e
Capture timing information while sorting
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-05-13 19:47:18 +10:00
Charles Korn 036c87223c
Ensure series in matrix values returned for instant queries are always sorted
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-05-13 11:03:15 +10:00
Charles Korn a4381608a0
Add failing test case
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-05-13 11:03:14 +10:00
Bryan Boreham ea82b49c33 [ENHANCEMENT] PromQL: use Kahan summation for sum()
This can give a more precise result, by keeping a separate running
compensation value to accumulate small errors.

See https://en.wikipedia.org/wiki/Kahan_summation_algorithm

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-09 14:29:38 +01:00
Arve Knudsen aabe4d6e4a promql.ActiveQueryTracker: Unmap mmapped file when done
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-09 14:27:29 +02:00
Bryan Boreham 3fd24d1cd7
Merge pull request #13999 from bboreham/extract-promqltest
[Test] Extract most PromQL test code into separate packages
2024-05-09 13:23:11 +01:00
Bryan Boreham 786e0e706c test: PromQL: stop using internal fields of engine
* set enablePerStepStats and lookback duration via
  `NewTestEngine` parameters.
* check maxSamples by recreating query engine
* check lookback without modifying internals

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 19:58:41 +01:00
Bryan Boreham 0dbfd20b69 test: move most PromQL tests into separate test package
So that they can import promqltest which imports promql.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:28:56 +01:00
Bryan Boreham 4a72607c4a refactor: extract some PromQL Engine tests which use unexported structs
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:27:17 +01:00
Bryan Boreham e7c77f7b40 promql: export NewTestQuery
So that tests can call it from another package.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:08:04 +01:00
Bryan Boreham babfcfdd91 refactor: Move NewTestEngine into promqltest
And export `DefaultMaxSamplesPerQuery` so callers can replicate previous
behaviour.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:07:28 +01:00
Oleg Zaytsev dabd789fd5
Quote label name in matchers when needed
When the label name of a matcher contains non-standard characters, like
a dot, or starts with a digit, it should be quoted.

If it's not quoted, then `VectorSelector.String()` isn't a valid PromQL.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-05-08 16:58:51 +02:00
Bryan Boreham b3c15d2246 test: clean up promqltest package references
So it nearly compiles.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham 9aa321d672 test: make field initializers explicit
Lint started complaining after I moved the file.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham ccf73266f0 test: move promqltest tests together with the implementation
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham 11b27d5d22 test: move test files into new promqltest package
So that promql package does not bring in test-only dependencies.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham a6e427660a test: check for @-modifier without using engine internals
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham 2b0c87b1b6 test: turn TestKahanSum into scripted test
This saves having a function solely to call kahanSumInc.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham a1af3c27d4 refactor: extract almost.Equal() to new package
To avoid a circular reference between promql and promqltest.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:10 +01:00
Arve Knudsen a25160e6a4
[REFACTOR] PromQL: simplify rangeEvalTimestampFunctionOverVectorSelector (#14021)
The function `rangeEvalTimestampFunctionOverVectorSelector` appeared to be checking histogram size, however the value it used was always 0 due to subtle variable shadowing.
However we don't need to pass sample values to the `timestamp` function, since the latter only cares about timestamps. This also affects peak sample count in statistics, since we are no longer copying histogram samples.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-08 10:39:44 +01:00
Oleg Zaytsev 4b7a44c7a0
Fix parser.VectorSelector.String() with empty name matcher (#14015)
The check fell into "this matcher equals vector selector's name" case when vector selector doesn't have a name and the matcher is an explicit matcher for an empty __name__ label.

To provide some context about why this is important: some downstream projects use the promql.Parse(expr.String()) to clone an expression's AST, and with this bug that matcher disappears in the cloning.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-05-06 11:51:08 +02:00
guangwu 9fda9443d4
fix(promql/query_logger): close file in error handling (#13948)
Signed-off-by: guoguangwu <guoguangwug@gmail.com>
2024-04-30 10:47:10 +02:00
Owen Williams 4a6f8704ef
parser: remake generated_parser output (#13923)
In a previous PR, the generated parser was created using an old version of goyacc.

Also adds -l to disable line directives, which fixes debug processing and reduces diffs at the expense of making it more difficult to reason about the generated output.

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-04-13 12:59:54 +02:00
Neeraj Gartia 612de026da
Adds Inf and NaN as Numbers to Histogram in Promql Testing Framework (#13916)
includes Inf and NaN as numbers to histogram

---------

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
Signed-off-by: Björn Rabenstein <github@rabenste.in>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
2024-04-11 12:53:28 +02:00
Matthieu MOREL 6f595c6762
golangci-lint: enable whitespace linter (#13905)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-04-11 09:27:54 +01:00
Bryan Boreham 12961c6a37 promql: refactor: eliminate one 'else'
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 0ac927515b promql: move group-seen into group struct
Save allocating an auxilliary array.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham 7499d90913 promql: remove pointer to aggregation groups
Just allocate in one slice.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00