Commit graph

830 commits

Author SHA1 Message Date
Jeanette Tan 8035c04624 Merge remote-tracking branch 'upstream/main'
Minor conflicts:
rules/manager.go
tsdb/compact.go
tsdb/db.go
go.mod
2023-07-19 21:40:27 +08:00
Julien Pivotto 0a48f93111
Merge pull request #10367 from ianwoolf/pr_add_close_for_query_logger
add Close for ActiveQueryTracker to close the file.
2023-07-18 13:53:18 +02:00
cui fliter 096ceca44f
remove repetitive words (#12556)
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-07-13 15:53:40 +02:00
beorn7 162612ea86 histograms: Improve comment
Oversight during review of #12525.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-12 14:52:49 +02:00
Ziqi Zhao 42d9169ba1 enhance histogram_quantile to get min/max value
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-07-12 04:29:54 +08:00
Carrie Edwards 2f9bc98b8a Add tests for min and max functions
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2023-07-11 21:51:20 +08:00
Carrie Edwards bc0ee4a469 Implement native histogram min and max query functions
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2023-07-11 21:51:20 +08:00
Bryan Boreham ce153e3fff Replace sort.Sort with faster slices.SortFunc
The generic version is more efficient.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-10 09:43:45 +00:00
Marco Pracucci 7ad111b27e
Merge remote-tracking branch 'remotes/prometheus/main' into update-upstream 2023-07-06 15:13:54 +02:00
Filip Petkovski 9949d83331 Add nolint for NewParser function
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-07-04 15:01:01 +00:00
Filip Petkovski df6e388d53 parser: Allow parsing arbitrary functions
In Thanos we would like to start experimenting with custom functions that are
currently not part of the PromQL spec. We would do this by adding an implementation
for those functions in the Thanos engine: https://github.com/thanos-community/promql-engine and allow
users to decide which engine they want to use on a per-query basis.

Since we use the PromQL parser from Prometheus, injecting functions in the global `Functions` variable
would mean they also become available for the Prometheus engine. To avoid this side-effect, this commit
exposes a Parser interface in which the supported functions can be injected as an option. If not functions
are injected, the parser implementation will default to the functions defined in the global Functions variable.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-07-04 15:01:01 +00:00
Bryan Boreham 7defd025cb Placate lint
I think the version using scoping was better, but I'm out of energy to fight the linter.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-04 15:01:01 +00:00
Bryan Boreham cfbbd2ce2a promql: include parsing in active-query tracking
So that the max-concurrency limit is applied.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-04 15:01:01 +00:00
Bryan Boreham 1706264cef promql: refactor: create query object before parsing
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-04 15:01:01 +00:00
Bryan Boreham 621d29795d promql: refactor: extract fn to wait on concurrency limit
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-04 15:01:01 +00:00
Giedrius Statkevičius 3f230fc9f8 promql: convert QueryOpts to interface
Convert QueryOpts to an interface so that downstream projects like
https://github.com/thanos-community/promql-engine could extend the query
options with engine specific options that are not in the original
engine.

Will be used to enable query analysis per-query.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
2023-07-03 16:20:31 +03:00
Charles Korn ad25a5c65f
Add test to confirm timestamp() behaves correctly when evaluating a range query. 2023-06-29 13:34:26 +10:00
Charles Korn 30507ef298
Create per-series iterators only once per selector, rather than recreating it for each time step.
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-06-28 15:41:19 +10:00
Charles Korn ab5d9e3be1
Don't create a new iterator for every time step.
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-06-28 15:41:19 +10:00
Charles Korn f5bbe2ef9e
Simplify loop
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-06-28 15:41:17 +10:00
Charles Korn 90ac889e1f
Expand series set just once
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-06-28 15:41:15 +10:00
Charles Korn 17b1486964
Inline method
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-06-28 15:41:14 +10:00
Charles Korn 28097c9476
Extract timestamp special case to its own method.
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-06-28 15:41:12 +10:00
Charles Korn 8bdcad30a4
Add benchmark for query using timestamp()
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-06-28 15:41:10 +10:00
Julien Pivotto a605b81b14
Merge pull request #12170 from fpetkovski/parser-inject-functions
parser: Allow parsing arbitrary functions
2023-06-27 13:32:46 +02:00
Bryan Boreham 67d2ef004d Placate lint
I think the version using scoping was better, but I'm out of energy to fight the linter.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-06-01 18:36:34 +00:00
Bryan Boreham bb0d8320dd promql: include parsing in active-query tracking
So that the max-concurrency limit is applied.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-06-01 18:16:05 +00:00
Bryan Boreham 71fc4f1516 promql: refactor: create query object before parsing
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-06-01 17:54:17 +00:00
Bryan Boreham 1f3821379c promql: refactor: extract fn to wait on concurrency limit
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-06-01 17:17:04 +00:00
Jeanette Tan 1be0816b46 Merge remote-tracking branch 'upstream/main' 2023-05-23 00:20:36 +08:00
zenador 191bf9055b
Handle more arithmetic operators for native histograms (#12262)
Handle more arithmetic operators and aggregators for native histograms

This includes operators for multiplication (formerly known as scaling), division, and subtraction. Plus aggregations for average and the avg_over_time function.

Stdvar and stddev will (for now) ignore histograms properly (rather than counting them but adding a 0 for them).

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-05-16 21:15:20 +02:00
beorn7 9e500345f3 textparse/scrape: Add option to scrape both classic and native histograms
So far, if a target exposes a histogram with both classic and native
buckets, a native-histogram enabled Prometheus would ignore the
classic buckets. With the new scrape config option
`scrape_classic_histograms` set, both buckets will be ingested,
creating all the series of a classic histogram in parallel to the
native histogram series. For example, a histogram `foo` would create a
native histogram series `foo` and classic series called `foo_sum`,
`foo_count`, and `foo_bucket`.

This feature can be used in a migration strategy from classic to
native histograms, where it is desired to have a transition period
during which both native and classic histograms are present.

Note that two bugs in classic histogram parsing were found and fixed
as a byproduct of testing the new feature:

1. Series created from classic _gauge_ histograms didn't get the
   _sum/_count/_bucket prefix set.
2. Values of classic _float_ histograms weren't parsed properly.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-05-13 01:32:25 +02:00
Justin Lei 7bbf24b707 Make MemoizedSeriesIterator not implement chunkenc.Iterator
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-05-03 12:45:39 -07:00
Justin Lei 6985dcbe73 Optimize and test MemoizedSeriesIterator
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-05-02 08:53:18 -07:00
Jeanette Tan 0fccba0db9 Merge remote-tracking branch 'upstream/main' 2023-04-26 21:25:21 +08:00
Matthieu MOREL 7e9acc2e46
golangci-lint: remove skip-cache and restore singleCaseSwitch rule
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-04-20 18:43:51 +02:00
Julien Pivotto f7c6130ff2
Merge pull request #12251 from prymitive/query_samples_total
Add query_samples_total metric
2023-04-20 15:48:24 +02:00
Matthieu MOREL bae9a21200
Merge branch 'main' into linter/nilerr
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-04-19 19:56:39 +02:00
beorn7 5b53aa1108 style: Replace else if cascades with switch
Wiser coders than myself have come to the conclusion that a `switch`
statement is almost always superior to a statement that includes any
`else if`.

The exceptions that I have found in our codebase are just these two:

* The `if else` is followed by an additional statement before the next
  condition (separated by a `;`).
* The whole thing is within a `for` loop and `break` statements are
  used. In this case, using `switch` would require tagging the `for`
  loop, which probably tips the balance.

Why are `switch` statements more readable?

For one, fewer curly braces. But more importantly, the conditions all
have the same alignment, so the whole thing follows the natural flow
of going down a list of conditions. With `else if`, in contrast, all
conditions but the first are "hidden" behind `} else if `, harder to
spot and (for no good reason) presented differently from the first
condition.

I'm sure the aforemention wise coders can list even more reasons.

In any case, I like it so much that I have found myself recommending
it in code reviews. I would like to make it a habit in our code base,
without making it a hard requirement that we would test on the CI. But
for that, there has to be a role model, so this commit eliminates all
`if else` occurrences, unless it is autogenerated code or fits one of
the exceptions above.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-19 17:22:31 +02:00
beorn7 c3c7d44d84 lint: Adjust to the lint warnings raised by current versions of golint-ci
We haven't updated golint-ci in our CI yet, but this commit prepares
for that.

There are a lot of new warnings, and it is mostly because the "revive"
linter got updated. I agree with most of the new warnings, mostly
around not naming unused function parameters (although it is justified
in some cases for documentation purposes – while things like mocks are
a good example where not naming the parameter is clearer).

I'm pretty upset about the "empty block" warning to include `for`
loops. It's such a common pattern to do something in the head of the
`for` loop and then have an empty block. There is still an open issue
about this: https://github.com/mgechev/revive/issues/810 I have
disabled "revive" altogether in files where empty blocks are used
excessively, and I have made the effort to add individual
`// nolint:revive` where empty blocks are used just once or twice.
It's borderline noisy, though, but let's go with it for now.

I should mention that none of the "empty block" warnings for `for`
loop bodies were legitimate.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-19 17:10:10 +02:00
Đurica Yuri Nikolić d162bb51b6
Merge pull request #485 from grafana/yuri/bring-prometheus-upstream
Synch with Prometheus up to 2023-04-18 (b028112)
2023-04-18 15:07:26 +02:00
Ben Ye fd3630b9a3 add ctx to QueryEngine interface
Signed-off-by: Ben Ye <benye@amazon.com>
2023-04-17 21:32:38 -07:00
Jeanette Tan 8bf0d6f59c Merge branch 'main' into sync-upstream-14-apr-2023 2023-04-18 10:10:09 +08:00
Marco Pracucci c461e22341
Improve fast regexp matcher cache (#482)
* Limit FastRegexMatcher by size (bytes) and add a TTL to each entry

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Add TestNewFastRegexMatcher_CacheSizeLimit

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Tolerate ristretto goroutines when checking goroutine leaks

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Tolerate ristretto goroutines when checking goroutine leaks

Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-04-17 15:20:58 +02:00
Jeanette Tan 894f657c48 Fix bugs from merge 2023-04-14 18:23:02 +08:00
Jeanette Tan 1570114ae1 Merge remote-tracking branch 'upstream/main' 2023-04-14 17:34:40 +08:00
ianwoolf 79e4bdee8e add Close for ActiveQueryTracker to close the file.
Signed-off-by: ianwoolf <btw515wolf2@gmail.com>
2023-04-14 14:43:23 +08:00
Matthieu MOREL fb3eb21230 enable gocritic, unconvert and unused linters
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-04-13 19:20:22 +00:00
beorn7 551de0346f promql: Do not return nil slices to the pool
Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-13 19:25:24 +02:00
beorn7 817a2396cb Name float values as "floats", not as "values"
In the past, every sample value was a float, so it was fine to call a
variable holding such a float "value" or "sample". With native
histograms, a sample might have a histogram value. And a histogram
value is still a value. Calling a float value just "value" or "sample"
or "V" is therefore misleading. Over the last few commits, I already
renamed many variables, but this cleans up a few more places where the
changes are more invasive.

Note that we do not to attempt naming in the JSON APIs or in the
protobufs. That would be quite a disruption. However, internally, we
can call variables as we want, and we should go with the option of
avoiding misunderstandings.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-13 19:25:24 +02:00