Commit graph

1118 commits

Author SHA1 Message Date
Charles Korn 6548421f61
promqltest: improve error messages when unexpected type is returned, and fix inverted expected and actual values in error message when scalar value does not match expected
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-16 13:37:05 +10:00
Björn Rabenstein 1daf7cdd62
Merge pull request #14626 from cuiweiyuan/main
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
chore: fix some function names
2024-08-15 11:46:21 +02:00
cuiweiyuan 1800af54f0 chore: fix some function names
Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>
2024-08-15 13:57:21 +08:00
Charles Korn 52818a97e2
Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms
# Conflicts:
#	promql/promqltest/testdata/native_histograms.test
2024-08-14 07:52:08 +10:00
Björn Rabenstein c2bc6cfe97
Merge pull request #14621 from charleskorn/panic-message
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
promql: clarify error message logged when panic occurs during query evaluation
2024-08-13 23:02:43 +02:00
George Krajcsovits fe5354df88
Merge pull request #14658 from krajorama/zero-rate-nh-quantile
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Native histograms: define behavior when rate is null.
2024-08-13 16:05:43 +02:00
György Krajcsovits 386fc8b9f6 Update from review comments.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-13 15:26:07 +02:00
Arve Knudsen 0503d4f372 PromQL: Fix comment regarding non-nil histogram pointer
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-08-13 08:55:24 +02:00
György Krajcsovits 6aee5b4b38 fix typo
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-12 12:04:45 +02:00
György Krajcsovits 06a8886b94 Native histograms: define behavior when rate is null.
Histogram quantile returns NaN in this case, which might be
surprising, so add a unit test that clarifies that this is
intentional.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-12 10:40:21 +02:00
suntala fd2f44af7f Fall back to comparing by label set when sorting by label desc
Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com>
Signed-off-by: suntala <arati.rana@grafana.com>
2024-08-11 21:44:03 +02:00
suntala 94ad489328 Fall back to comparing by label set when sorting by label
Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com>
Signed-off-by: suntala <arati.rana@grafana.com>
2024-08-11 21:44:03 +02:00
Charles Korn f992f81bd0
Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms
Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com>
2024-08-09 13:58:54 +10:00
Charles Korn 5cfdde327c
Address PR feedback: add extra test case
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-09 13:57:37 +10:00
Charles Korn 82bb35fabb
Address PR feedback: fix typo and rename variable
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-09 13:51:31 +10:00
Björn Rabenstein 41f8a0223c
Merge pull request #14609 from charleskorn/rate-over-mixed-custom-exponential-histograms
promql: fix "cannot reduce resolution to custom buckets schema" panic in `rate` over native histograms with mix of custom and exponential buckets
2024-08-08 14:03:41 +02:00
Manik Rana 02c465bf58
textparse: Implement CreatedTimestamp() in openmetricsparse.go (#14356)
* feat: initial implement of createedTimestamp() with tests

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: return ct after finding it

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: remove unneeded test

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: add comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: multiple changes

- implement changes from pair programming session
- use newParse.val()
- advance parser p if ct is found

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: check if err from p.Next()

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: advance parser and parse histograms + summary

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: restore previous tests

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: retore failing tests

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: remove unneeded comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: return nil when mtype doesn't match

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: update go fmt version

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: cleanup

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: document deepcopyparser

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: cover edgecase of `gauge_created` in CreatedTimestamp()

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: readability updates

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: dedeuplicate labeldiff checks

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* tests: add tests for new label functions

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: document CreatedTimestamp func

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: optimize `CreatedTimestamp()`

 - Use refactored CreatedTimestamp function with bug fixes
 - Remove unused code in labels.go
 - Improve code documentation

 Signed-off-by: Manik Rana <manikrana54@gmail.com>

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: add tests and lint fixes

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: remove mName

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* tests: add tests for CT parse failures and deepCopy

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: edit expectCT struct

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: add new label in deepCopy

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: use p.builder in deepCopy

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: add NewMetricsParserWithOpts

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* chore: comments

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* chore: rename var

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: add condition for OM fuzzing

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: build tags

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: default skipCT to false

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: rename skipCT to skipCTSeries

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: formatting

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments and readability updates

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: remove NewOpenMetricsParserWithOpts

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: extract skipCTSeries logic from parseMetricSuffix

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: inline create a NewOpenMetricsParser

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: improve error handling

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: return error instead of nil

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: remove skipCT check from tBraceOpen

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* Pair programming with Manik, Arthur and Daniel.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* chore: comments and use helper funcs

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

---------

Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-08-08 12:35:35 +01:00
Charles Korn f91009aa2e
promql: clarify error message when panic occurs during query evaluation
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-08 09:11:38 +10:00
Björn Rabenstein 27579c9148
Merge pull request #14605 from krajorama/fix-staleness-pool-corrupt
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Fix histogram pool poisoning bug chunkenc.Iterator
2024-08-07 21:02:08 +02:00
George Krajcsovits 17b0b788da
Update promql/engine.go
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-08-07 20:15:46 +02:00
Charles Korn 0f4bc87b4f
Make linter happy
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-07 15:35:06 +10:00
Charles Korn f07b3ae67b
Fix issue where avg over mixed exponential and custom buckets, or incompatible custom buckets, produces incorrect results or panics
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-07 15:32:35 +10:00
Charles Korn 5ee94f49a2
Fix issue where sum over mixed exponential and custom buckets, or incompatible custom buckets, produces incorrect results
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-07 15:30:01 +10:00
Charles Korn 424cefcf5e
Fix "cannot reduce resolution to custom buckets schema" panic in rate over native histograms with mix of custom and exponential buckets
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-07 14:45:02 +10:00
Charles Korn f6e4b775e2
Check for errors first
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-07 14:25:46 +10:00
György Krajcsovits b91acc61b0 Add unit test
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-06 20:50:51 +02:00
Björn Rabenstein ee5bba07c0
Merge pull request #14413 from prometheus/beorn7/promql
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
promql: more Kahan summation (avg) and less incremental mean calculation (avg, avg_over_time)
2024-08-06 19:56:32 +02:00
Björn Rabenstein 71ba554293
Merge pull request #14482 from ywwg/owilliams/group-quote
promql: support quoting in grouping label lists
2024-08-06 19:43:38 +02:00
György Krajcsovits 37c8c9257b Fix histogram pool poisoning bu chunkenc.Iterator
chunkenc.Iterator.AtFloatHistogram may do a shallow copy if
it receives nil as input pointer. This can in turn share the
span slice with multiple histograms in the matrixSelectorHPool,
leading to unexpected errors.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-06 19:40:14 +02:00
Charles Korn aadec25faf
promql: Fix issue where some native histogram-related annotations are not emitted by rate (#14575)
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-06 09:10:40 +01:00
Owen Williams d90c5a71d7 support quoting in grouping label lists
Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-08-01 10:07:12 -04:00
Jan Fajerski 8a96a75ba8 Merge branch 'main' into 3.0-main-sync-24-08-01
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
2024-08-01 15:44:47 +02:00
Filip Petkovski 3e6155e596
Use CopyTo when resetting histogram in stats iterator
The histogram stats iterator does not fully clear the histogram object
and is not resilient to new fields being added to the histogram type.

To resolve the issue, the commit uses the CopyTo methods which should
be future proof to new fields being added.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2024-07-31 11:18:47 +02:00
Charles Korn 7fab72a280
promqltest: add support for setting counter reset hint on histogram samples (#14537)
* promqltest: add support for setting counter reset hint on histogram samples

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-07-31 09:53:05 +02:00
Max Amin 84b819a69f
feat: add Google cloud roundtripper for remote write (#14346)
* feat: Google Auth for remote write

Signed-off-by: Max Amin <maxamin@google.com>

---------

Signed-off-by: Max Amin <maxamin@google.com>
2024-07-30 16:25:19 +01:00
Filip Petkovski 2cd97c61e0
Add more test cases
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2024-07-29 14:53:32 +02:00
Arve Knudsen 7c873004c7 Merge remote-tracking branch 'prometheus/main' into arve/close-engine 2024-07-26 11:48:33 +02:00
Filip Petkovski be7a4c9b83
Ignore stale histograms for counter reset detection
The histogram stats decoder keeps track of the last seen histogram sample
in order to properly detect counter resets. We are seeing an issue where
a histogram with UnknownResetHint gets treated as a counter reset when it follows
a stale histogram sample.

I believe that this is incorrect since stale samples should be completely ignored
in PromQL. As a result, they should not be stored in the histogram stats iterator
and the counter reset detection needs to be done against the last non-stale sample.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2024-07-26 10:08:31 +02:00
Jan Fajerski adf5d6bce1 Merge branch 'main' into 3.0-main-sync-24-07-18
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>

Conflicts:
	VERSION
          pick 3.0.0
	promql/promqltest/testdata/histograms.test
          pick changes from c39776c5b5,
          but adjust 5m range selectors to 10m to account for
          https://github.com/prometheus/prometheus/pull/13904.
Fixes:
        promql/promqltest/testdata/functions.test
	promql/promqltest/testdata/staleness.test
          Tests added in https://github.com/prometheus/prometheus/pull/9138
          need to be adjusted to account for
          https://github.com/prometheus/prometheus/pull/13904.
2024-07-18 15:56:40 +02:00
Björn Rabenstein ca7062cf49
Merge pull request #14464 from prometheus/beorn7/histogram
promql: Add NHCB tests
2024-07-17 14:57:46 +02:00
beorn7 c39776c5b5 promql: Add NHCB tests
This adds equivalent NHCB tests to the existing classic histogram
tests.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-07-16 12:20:43 +02:00
Bryan Boreham bb241d7960
Merge pull request #14463 from fpetkovski/customize-acceptance-tests
Allow built-in tests to be customized
2024-07-15 10:04:41 +01:00
Arve Knudsen fbc9eddfaf Refactor engine creation in tests
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-07-14 13:58:51 +02:00
Arve Knudsen fec6adadcd Merge remote-tracking branch 'prometheus/main' into arve/close-engine 2024-07-14 13:19:11 +02:00
Bryan Boreham d116bf7b9f
Merge pull request #14109 from harry671003/pass_limit_to_querier
storage: pass limit param as hint in querier
2024-07-12 10:27:52 +01:00
Filip Petkovski cb8b6d7504
Allow built-in tests to be customized
The RunBuiltinTests function accepts a concrete type which makes
it hard to exclude certain tests from the suite. It would be great
if we could skip tests which might not be critical in order to unblock
updates.

By accepting an interface instead, we can inject a custom implementation
which would skips select test cases.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2024-07-11 14:19:09 +02:00
beorn7 cff0429b1a promql: make avg_over_time faster and more precise
Same idea as for the avg aggregator before: Most of the time, there is
no overflow, so we don't have to revert to the more expensive and less
precise incremental calculation of the mean value.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-07-10 19:20:24 +02:00
beorn7 3a908d8e08 promql: Improve Kahan usage in avg_over_time
The calculation of the mean value in avg_over_time is performed in an
incremental fashion. This introduces additional numerical errors that
even Kahan summation cannot compensate, but at least we can use the
Kahan-corrected mean value when we use the intermediate mean value in
the calculation.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-07-10 19:20:24 +02:00
beorn7 c46074f4dd promql: make avg aggregation more precise and less expensive
The basic idea here is that the previous code was always doing
incremental calculation of the mean value, which is more costly and
can be less precise. It protects against overflows, but in most cases,
an overflow doesn't happen anyway.

The other idea applied here is to expand on #14074, where Kahan
summation was applied to sum().

With this commit, the average is calculated in a conventional way
(adding everything up and divide in the end) as long as the sum isn't
overflowing float64. This is combined with Kahan summation so that the
avg aggregation, in most cases, is really equivalent to the sum
aggregation with a following division (which is the user's expectation
as avg is supposed to be syntactic sugar for sum with a following
divison).

If the sum hits ±Inf, the calculation reverts to incremental
calculation of the mean value. Kahan summation is also applied here,
although it cannot fully compensate for the numerical errors
introduced by the incremental mean calculation. (The tests added in
this commit would fail if incremental mean calculation was always
used.)

Signed-off-by: beorn7 <beorn@grafana.com>
2024-07-10 19:20:24 +02:00
darshanime bd4ea118e9 Allow durations for number rule
Signed-off-by: darshanime <deathbullet@gmail.com>
2024-07-10 15:50:22 +02:00
darshanime cfad8ff3b2 Deprecate duration token
Signed-off-by: darshanime <deathbullet@gmail.com>
2024-07-10 15:49:58 +02:00
darshanime 8c8860d2d6 Allow number literals as duration
Signed-off-by: darshanime <deathbullet@gmail.com>
2024-07-10 15:49:30 +02:00
Filip Petkovski acb6c1ae4b
Fix decoding buckets for native histograms in binops
The optimizer which detects cases where histogram buckets can be skipped
does not take into account binary expressions. This can lead to buckets
not being decoded if a metric is used with both histogram_fraction/quantile and
histogram_sum/count in the same expression.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2024-07-10 11:55:29 +02:00
Jan Fajerski c0d67fd845 Merge branch 'main' into 3.0-main-sync-24-07-09 2024-07-09 16:51:07 +02:00
beorn7 b5b04ddbe3 promql: add avg aggregation benchmark
Signed-off-by: beorn7 <beorn@grafana.com>
2024-07-04 18:59:30 +02:00
beorn7 44d8c1d182 nit: add period at end of sentence
Signed-off-by: beorn7 <beorn@grafana.com>
2024-07-04 17:31:39 +02:00
beorn7 9a837b7f3c promql: Make groupedAggregation.groupCount a float64
It's always used as such. Let's avoid the countless conversions.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-07-04 17:31:34 +02:00
JuanJo Ciarlante c94c5b64c3
feat: add limitk() and limit_ratio() operators (#12503)
* rebase 2024-07-01, picks previous renaming to `limitk()` and `limit_ratio()`

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* gofumpt -d -extra

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* more lint fixes

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* more lint fixes+

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* put limitk() and limit_ratio() behind --enable-feature=promql-experimental-functions

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* EnableExperimentalFunctions for TestConcurrentRangeQueries() also

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* use testutil.RequireEqual to fix tests, WIP equivalent thingie for require.Contains

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* lint fix

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* moar linting

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* rebase 2024-06-19

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* re-add limit(2, metric) testing for N=2 common series subset

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* move `ratio = param` to default switch case, for better readability

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* gofumpt -d -extra util/testutil/cmp.go

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* early break when reaching k elems in limitk(), should have always been so (!)

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* small typo fix

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* no-change small break-loop rearrange for readability

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* remove IsNan(ratio) condition in switch-case, already handled as input validation

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* no-change adding some comments

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* no-change simplify fullMatrix() helper functions used for tests

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* add `limitk(-1, metric)` testcase, which is handled as any k < 1 case

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* engine_test.go: no-change create `requireCommonSeries() helper func (moving code into it) for readability

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* rebase 2024-06-21

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* engine_test.go: HAPPY NOW about its code -> reorg, create and use simpleRangeQuery() function, less lines and more readable ftW \o/

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* move limitk(), limit_ratio() testing to promql/promqltest/testdata/limit.test

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* remove stale leftover after moving tests from engine_test.go to testdata/

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* fix flaky `limit_ratio(0.5, ...)` test case

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* Update promql/engine.go

Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* Update promql/engine.go

Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* Update promql/engine.go

Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* fix AddRatioSample() implementation to use a single conditional (instead of switch/case + fallback return)

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* docs/querying/operators.md: document r < 0

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* add negative limit_ratio() example to docs/querying/examples.md

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* move more extensive docu examples to docs/querying/operators.md

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* typo

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* small docu fix for poor-mans-normality-check, add it to limit.test ;)

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* limit.test: expand "Poor man's normality check" to whole eval range

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* restore mistakenly removed existing small comment

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* expand poors-man-normality-check case(s)

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* Revert "expand poors-man-normality-check case(s)"

This reverts commit f69e1603b2ebe69c0a100197cfbcf6f81644b564, indeed too
flaky 0:)

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* remove humor from docs/querying/operators.md

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* fix signoff

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* add web/ui missing changes

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* expand limit_ratio test cases, cross-fingering they'll not be flaky

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* remove flaky test

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* add missing warnings.Merge(ws) in instant-query return shortcut

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* add missing LimitK||LimitRatio case to codemirror-promql/src/parser/parser.ts

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* fix ui-lint

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

* actually fix returned warnings :]

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>

---------

Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
2024-07-03 22:18:57 +02:00
Arve Knudsen 493e196f10 Fix bad merge
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-07-03 15:12:40 +02:00
Arve Knudsen 020b0f200a Merge remote-tracking branch 'prometheus/main' into arve/close-engine 2024-07-03 15:12:12 +02:00
Jan Fajerski b4152309a4 Merge branch 'main' into 3.0-main-sync
Conflicts:
  promql/engine_test.go
    Resolved by picking main changes but adjusting total_samples for
    query "max_over_time(metricWith1HistogramEvery10Seconds[60s])[20s:5s]"
    to 312. Via https://github.com/prometheus/prometheus/pull/13662 this
    histogram now stores 13 values per timestamp, but via
    https://github.com/prometheus/prometheus/pull/13904 the range query
    is now left-open.
  promql/promqltest/testdata/functions.test
    Resolved by picking changes in main. See also
    https://github.com/prometheus/prometheus/pull/13662, but adjust some
    range selectors (`s/1m/2m/`) to account for
    https://github.com/prometheus/prometheus/pull/13904.
  promql/promqltest/testdata/histograms.test
    Resolved by picking changes in main. See also
    https://github.com/prometheus/prometheus/pull/13662, but adjust some
    range selectors (`s/5m/10m/`) to account for
    https://github.com/prometheus/prometheus/pull/13904.

Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
2024-07-03 09:45:25 +02:00
Ganesh Vernekar 3d54bcc018
Merge pull request #14362 from charleskorn/charleskorn/sum-infinity 2024-07-03 01:05:03 -04:00
Arve Knudsen e8ae8cf012 Merge remote-tracking branch 'prometheus/main' into arve/close-engine
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-07-01 10:47:21 +02:00
Charles Korn fd6bdf5230
Fix issue where summation of +/- infinity returns NaN instead of infinity
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-28 11:26:54 +10:00
Björn Rabenstein 2e58d46522
Merge pull request #13662 from prometheus/nhcb
Native histograms custom buckets storage
2024-06-27 21:44:20 +02:00
Björn Rabenstein 19da579799
Merge pull request #12821 from rexagod/12769
parser: support underscores
2024-06-27 17:04:18 +02:00
Pranshu Srivastava 3c9e3ee552
parser: support underscores
Support underscores in numbers, namely, decimals, hexadecimals, and
exponentials.

Fixes #12769
Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>

Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>
2024-06-27 19:20:52 +05:30
Bryan Boreham b6aba4ff14
Merge pull request #14074 from bboreham/kahan-sum-sum
[ENHANCEMENT] PromQL: use Kahan summation for sum()
2024-06-24 11:13:26 +01:00
🌲 Harry 🌊 John 🏔 d5f6887294 Pass limit param as hint to storage.Querier
Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>
2024-06-20 09:47:38 -07:00
Jeanette Tan dda5f48c9e Merge branch 'main' into nhcb-review-2 2024-06-20 22:50:00 +08:00
Jeanette Tan fc9dc72028 remove eval_with_nhcb
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-20 22:49:00 +08:00
Jeanette Tan a6d788b1be update missed suggested change from code review
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-20 22:49:00 +08:00
Zhang Zhanpeng debbdb8608 make matrix selection and lookback left-open and right-closed
Signed-off-by: Zhang Zhanpeng <zhangzhanpeng.zzp@alibaba-inc.com>
Signed-off-by: beorn7 <beorn@grafana.com>
Co-authored-by: beorn7 <beorn@grafana.com>
2024-06-20 22:05:40 +08:00
Bryan Boreham 84602bbace
Merge branch 'main' into fix-matcher-string-with-empty-label-name 2024-06-19 05:56:25 -04:00
Charles Korn aeec30f082
Convert TestTimestampFunction_StepsMoreOftenThanSamples
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-17 16:56:56 +10:00
Charles Korn 987fa5c6a2
Convert range query test cases to test scripting language
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-17 16:43:01 +10:00
Arve Knudsen b7320ef636 Merge remote-tracking branch 'prometheus/main' into arve/close-engine 2024-06-14 10:51:35 +02:00
Oleg Zaytsev 03cf6141d4
Fix Matcher.String() with empty label name
When the label name is empty, which can happen now with quoted label
name, it should be quoted when printed as a string again.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-06-13 18:46:35 +02:00
Björn Rabenstein 08621bebe9
Merge pull request #14269 from prometheus/beorn7/histogram-test
promql: Add tests for histogram counter reset only in bucket
2024-06-08 16:59:47 +02:00
Jeanette Tan b8cb12b989 rename hist to histogram according to code review
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-07 20:26:41 +08:00
Jeanette Tan 4c2aa872d4 update readme for testing framework
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-07 20:21:05 +08:00
Jeanette Tan 14f8dded39 Merge branch 'main' into nhcb
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-07 19:17:14 +08:00
Jeanette Tan b0e320425c refine test rewriting regex and add validation
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-07 18:50:59 +08:00
Jeanette Tan 9adc1699c3 fix according to code review
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-07 18:50:59 +08:00
Charles Korn 1f988f77ff
promql: extend test scripting language to support asserting on expected error message (#14038)
Add ability to assert that a query fails with a particular error message

This also adds documentation for the test scripting language in general,
including the new feature.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-06 17:56:25 +02:00
beorn7 c7fdfe8004 promql: Add tests for histogram counter reset only in bucket
This also exercises the "fast path" (only decoding count and sum),
i.e. where the counter reset isn't visible at all in the decoded data.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-06-06 17:47:38 +02:00
Filip Petkovski 6e68046c25
Implement histogram statistics decoder (#14097)
Implement histogram statistics decoder

This commit speeds up histogram_count and histogram_sum
functions on native histograms. The idea is to have separate decoders which can be
used by the engine to only read count/sum values from histogram objects. This should help
with reducing allocations when decoding histograms, as well as with speeding up aggregations
like sum since they will be done on floats and not on histogram objects.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

---------

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
2024-06-06 17:17:13 +02:00
Arve Knudsen e57aac8084 Merge remote-tracking branch 'prometheus/main' into arve/close-engine
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-06-05 11:37:44 +02:00
Charles Korn 24ef000025
Merge branch 'main' into sort-matrix-series
# Conflicts:
#	promql/engine_test.go
2024-05-31 12:44:27 +10:00
Arve Knudsen 0cc99e677a promql.Engine: Add Close method
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-28 12:01:47 +02:00
Arve Knudsen f3b8750339 Join errors
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-27 17:14:17 +02:00
Arve Knudsen 7b56353090 Merge remote-tracking branch 'prometheus/main' into arve/query-logger-munmap 2024-05-27 17:08:33 +02:00
Björn Rabenstein 114dc5c393
Merge pull request #13638 from NeerajGartia21/promql-test
Converts existing native histogram unit tests to the PromQL testing framework
2024-05-19 19:35:36 +02:00
Arve Knudsen 0f01d4b336 Fix flaky test
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-15 21:58:56 +02:00
Arve Knudsen bf8d88f326 Merge remote-tracking branch 'origin/main' into arve/query-logger-munmap 2024-05-15 21:02:03 +02:00
Oleksandr Redko f10c3454e9 Enable perfsprint linter and fix up code
Signed-off-by: Oleksandr Redko <oleksandr.red+github@gmail.com>
2024-05-15 17:51:05 +03:00
Björn Rabenstein e6be4240be
Merge pull request #14068 from colega/quote-label-name-in-matchers-when-needed
Bugfix: quote label name in matchers when needed
2024-05-14 17:18:58 +02:00
Jeanette Tan f028496133 Merge branch 'main' into nhcb
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-05-14 16:20:15 +08:00
Neeraj Gartia 661856cb65 removes the added tests from engine_test.go
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:58:25 +05:30
Neeraj Gartia 6119124d0e some nits
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:55:38 +05:30