Commit graph

170 commits

Author SHA1 Message Date
Julien Pivotto d992c36b3a promql: make active query tracker context-aware (#6701)
* promql: make query logger context-aware
* Remove gate
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-27 22:29:44 +00:00
Julien Pivotto 5f27ac3583 Refactor query log fields (#6694)
* Refactor query log fields

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-27 09:53:10 +00:00
Tobias Guggenmos 2aacd807b3 PromQL: Various small improvements in the parser (#6652)
* Move check for empty VectorSelector to typeChecking
* Move check for twice set metric name to typeChecking
* Make child of MatrixSelector a general Node
* rename checkType to checkAST
* Rename fail to addParseErr
* Remove trailing whitespace

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-01-17 15:16:58 +00:00
Tobias Guggenmos 3a204be6b7 PromQL: Fix string and parentheses handling in engine (#6612)
* WIP: PromQL: Allow engine to return strings

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Add test suggested by @roidelapluie

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Fix typo in React UI

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Fix parenthesis handling for functions and aggregator params

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Add more tests

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Fix React UI test

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-01-15 18:31:58 +01:00
Tobias Guggenmos 0c8e9ef09e PromQL: Add position metadata to the AST (#6615)
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
2020-01-14 16:12:15 +00:00
Tobias Guggenmos 64194f7d45 PromQL: AST: Make VectorSelector Children of MatrixSelector (#6590)
Make Vector selectors children of Matrix Selectors

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-01-10 14:25:41 +00:00
Julien Pivotto 3885562587 Query Logging styling (#6594)
- Fix Json vs JSON in activequerylogger
- Fix SetQueryLogger always returns nil

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-09 21:11:39 +00:00
Julien Pivotto 9d9bc524e5 Add query log (#6520)
* Add query log, make stats logged in JSON like in the API

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-08 13:28:43 +00:00
Julien Pivotto e0afec906f add absent_over_time (#6490)
* Implement absent_over_time

Fixes #2882

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-03 15:26:12 +00:00
Mark Nevill b0a5c51b95 Return unused point slice to pool in Call and VectorSelector eval. (#6427)
Signed-off-by: Mark Nevill <mark.nevill@gmail.com>
2019-12-09 10:32:40 +00:00
Garrett 5a9c4acfbf Pushdown aggregator group by through read hint (#6401)
* Pushdown aggregator group by through read hint

Implement https://github.com/prometheus/prometheus/issues/6400

* add temporal aggregation pushdown support

Signed-off-by: xiancli <xiancli@ebay.com>
2019-12-05 14:06:28 +00:00
Tobias Guggenmos bbd92b85da promql: Use capitalized names for item types (#6371)
For yacc generated parsers there is the convention to capitalize the names of item types provided by the lexer, which makes it easy to distinct lexer tokens (capitalized) from nonterminal symbols (not capitalized) in language grammars.

This convention is also followed by the (non generated) go compiler (see https://golang.org/pkg/go/token/#Token).

Part of the parser rewrite described in #6256.

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2019-11-26 13:29:42 +00:00
AllenZMC ead0933dd9 fix word 'substracting' to 'subtracting' (#5822)
Signed-off-by: czm <zhongming.chang@daocloud.io>
2019-08-01 15:44:38 +01:00
Advait Bhatwadekar 5d401f1e1b Added query logging for prometheus. Issue #1315 (#5794)
* Added query logging for prometheus.
Options added:
1) active.queries.filepath: Filename where queries will be recorded
2) active.queries.filesize: Size of the file where queries will be recorded.

Functionality added:
All active queries are now logged in a file. If prometheus crashes unexpectedly, these queries are also printed out on stdout in the rerun.

Queries are written concurrently to an mmaped file, and removed once they are done. Their positions in the file are reused. They are written in json format. However, due to dynamic nature of application, the json has an extra comma after the last query, and is missing an ending ']'. There may also null bytes in the tail of file.

Signed-off-by: Advait Bhatwadekar <advait123@ymail.com>
2019-07-31 16:12:43 +01:00
Ganesh Vernekar 588eb20018
Efficient iteration and search in HashForLabels and HashWithoutLabels (#5707)
* Efficient iteration and search in Labels.HashForLabels

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Better names for variables

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* HashWithoutLabels optimizations

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor HashForLabels and HashWithoutLabels to take sorted names

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-06-28 18:22:51 +05:30
beorn7 dd81912554 Add objectives to Summaries
With the next release of client_golang, Summaries will not have
objectives by default. To not lose the objectives we have right now,
explicitly state the current default objectives.

Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-12 02:03:13 +02:00
Thomas Jackson a000cec011 Re-use label builder in promql aggregation (#5641)
For my benchmarks on aggregation this reduces allocations by ~5% (~10%
time improvement):

```
benchmark                                                    old ns/op     new ns/op     delta
BenchmarkEvaluations/benchdata/aggregators.test/promxy-4     727692        649626        -10.73%

benchmark                                                    old allocs     new allocs     delta
BenchmarkEvaluations/benchdata/aggregators.test/promxy-4     2566           2434           -5.14%

benchmark                                                    old bytes     new bytes     delta
BenchmarkEvaluations/benchdata/aggregators.test/promxy-4     162760        148854        -8.54%
```

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>
2019-06-11 09:24:49 +01:00
Goutham Veeramachaneni 3cc5f9d880 Make sure subquery range is taken into account for selection (#5467)
* Make sure subquery range is taken into account for selection

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2019-04-17 13:52:41 +01:00
Julius Volz bc1c7f1809 Fix scalar-vector comparisons (#5454)
* Fix scalar-vector comparisons

Fixes https://github.com/prometheus/prometheus/issues/5452

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-04-11 10:42:16 +01:00
Bryan Boreham 69dd090880 Check for cancellation on every step of a range evaluation
Signed-off-by: Bryan Boreham <bryan@weave.works>
2019-04-10 13:27:45 +01:00
Bryan Boreham e4a37d0986 Replace select with simpler error check
The documentation for Context states that this is just as good:
	// If Done is not yet closed, Err returns nil.
	// If Done is closed, Err returns a non-nil error

Signed-off-by: Bryan Boreham <bryan@weave.works>
2019-04-10 13:27:45 +01:00
Tariq Ibrahim 8fdfa8abea refine error handling in prometheus (#5388)
i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors.
ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives.
iii) Does away with the use of fmt package for errors in favour of pkg/errors

Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-26 00:01:12 +01:00
Julius Volz 8155cc4992
Expose lexer item types (#5358)
* Expose lexer item types

We have generally agreed to expose AST types / values that are necessary
to make sense of the AST outside of the promql package. Currently the
`UnaryExpr`, `BinaryExpr`, and `AggregateExpr` AST nodes store the lexer
item type to indicate the operator type, but since the individual item
types aren't exposed, an external user of the package cannot determine
the operator type. So this PR exposes them.

Although not all item types are required to make sense of the AST (some
are really only used in the lexer), I decided to expose them all here to
be somewhat more consistent. Another option would be to not use lexer
item types at all in AST nodes.

The concrete motivation is my work on the PromQL->Flux transpiler, but
this ought to be useful for other cases as well.

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fix item type names in tests

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-03-14 20:53:55 +01:00
Daisy T 683fbc59ec exponentation operator to drop metric name in result of op operation (#5329)
Signed-off-by: Daisy T <daisyts@gmx.com>
2019-03-12 10:21:42 +00:00
Brian Brazil 858c363e94 Fix panic when aggregator param is not a literal.
The return value for checkForSeriesSetExpansion
is always nil, simplify.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-03-04 12:00:05 +00:00
Tariq Ibrahim a2a6e24f9f show list of offending labels in the error message in many-to-many scenarios (#5189)
Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-02-09 10:17:52 +01:00
Bryan Boreham 8841692a63 Use the context associated with the inner evaluation span (#5130)
Signed-off-by: Bryan Boreham <bryan@weave.works>
2019-01-28 18:33:30 +00:00
Matt Layher 43c9d9e91f promql: apply golint suggestions (#5066)
Signed-off-by: Matt Layher <mdlayher@gmail.com>
2019-01-08 18:26:02 +00:00
Simon Pasquier f678e27eb6
*: use latest release of staticcheck (#5057)
* *: use latest release of staticcheck

It also fixes a couple of things in the code flagged by the additional
checks.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Use official release of staticcheck

Also run 'go list' before staticcheck to avoid failures when downloading packages.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-04 14:47:38 +01:00
Tom Wilkie 6e08029b56
Move err to be the last return value from storage.Select. (#5054)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-01-02 11:10:13 +00:00
Ganesh Vernekar dbe55c1352 Subquery (#4831)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-12-22 13:47:13 +00:00
Tom Wilkie e1d9bf77f1
Export the error field in ErrStorage, so we can 'throw' it outside the package. (#4954)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-12-04 16:49:21 +00:00
mknapphrt f0e9196dca Return warnings on a remote read fail (#4832)
Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>
2018-11-30 14:27:12 +00:00
Ben Kochie c6399296dc
Fix spelling/typos (#4921)
* Fix spelling/typos

Fix spelling/typos reported by codespell/misspell.
* UK -> US spelling changes.

Signed-off-by: Ben Kochie <superq@gmail.com>
2018-11-27 17:44:29 +01:00
Bryan Boreham 9a956872a3 Make ErrorStorage a concrete type not an interface
Since it is used in a type assertion, having it as an alias to the
error interface is the same as saying 'error', i.e. it succeeds for
all types of error.  Change to a struct which is a concrete type and
the type assertion will only succeed if the type is identical.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2018-10-04 13:13:41 +00:00
Callum Styan 9bca041285 WIP: keep track of samples per query, set a max # of samples (#4513)
* keep track of samples per query, set a max # of samples that can be in
memory at once

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2018-10-02 12:59:19 +01:00
Tom Wilkie 4c52400708
Limit concurrent remote reads. (#4656)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-09-25 20:07:34 +01:00
Harsh Agarwal 18a9a390b5 Add duplicate-labelset check for range/instant vectors (#4589)
Signed-off-by: Harsh Agarwal <cs15btech11019@iith.ac.in>
2018-09-18 10:46:13 +01:00
Ganesh Vernekar 576ee4d309 Label name check for 'count_values' (#4585)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-13 15:27:36 +05:30
Dan Cech 9f4cb06a37 use Welford/Knuth method to compute standard deviation and variance (#4533)
* use Welford/Knuth method to compute standard deviation and variance, avoids float precision issues
* use better method for calculating avg and avg_over_time

Signed-off-by: Dan Cech <dcech@grafana.com>
2018-08-26 10:28:47 +01:00
Goutham Veeramachaneni 71855a22a4
Add tracing spans to promql (#4436)
* Add spans to promql

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Simplify timer and span tracking.

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2018-08-16 13:11:34 +05:30
Thomas Jackson 56daa1f28a Only add LookbackDelta to vector selectors (#4399)
Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>

Related to #4226
2018-07-19 06:16:05 +01:00
Alin Sinpalean 372e7652b7 Reuse (copy) overlapping matrix samples between range evaluation steps (#4315)
* Reuse (copy) overlapping matrix samples between range evaluation steps.

Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>
2018-07-18 11:14:02 +01:00
Tony Lee bcdaf8e2d2 add unused pointslices to the pool (#4363)
Signed-off-by: Tony Lee <tl@hudson-trading.com>
2018-07-18 05:29:21 +01:00
Alin Sinpalean e3b775b78b Simplify BufferedSeriesIterator usage (#4294)
* Allow for BufferedSeriesIterator instances to be created without an underlying iterator, to simplify their usage.

Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>
2018-07-18 05:10:28 +01:00
Julius Volz 219e477272 Fix some (valid) lint errors (#4287)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-07-18 05:07:33 +01:00
Thomas Jackson 92c6f0c92e Add offset to selectParams (#4226)
* Add Start/End to SelectParams
* Make remote read use the new selectParams for start/end

This commit will continue sending the start/end time of the remote read
query as the overarching promql time and the specific range of data that
the query is intersted in receiving a response to is now part of the
ReadHints (upstream discussion in #4226).

* Remove unused vendored code

The genproto.sh script was updated, but the code wasn't regenerated.
This simply removes the vendored deps that are no longer part of the
codegen output.

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>
2018-07-18 04:58:00 +01:00
Alin Sinpalean 96fb0b2155 Optimize PromQL aggregations (#4248)
* Compute hash of label subsets without creating a LabelSet first.

Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>
2018-07-18 04:56:27 +01:00
Tom Wilkie 3228814456 Don't forget to register query_duration_seconds{slice="queue_time"} (#4381)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-07-15 12:24:37 +01:00
Thomas Jackson a6dace8829 Check for timeout in each iteration of matrixSelector (#4300)
Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>

Fixes #4288
2018-06-21 22:43:31 +01:00