Commit graph

38 commits

Author SHA1 Message Date
Tobias Guggenmos 5c503d85f7 PromQL: export lexer (#6435)
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2019-12-09 19:03:31 +00:00
Tobias Guggenmos bbd92b85da promql: Use capitalized names for item types (#6371)
For yacc generated parsers there is the convention to capitalize the names of item types provided by the lexer, which makes it easy to distinct lexer tokens (capitalized) from nonterminal symbols (not capitalized) in language grammars.

This convention is also followed by the (non generated) go compiler (see https://golang.org/pkg/go/token/#Token).

Part of the parser rewrite described in #6256.

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2019-11-26 13:29:42 +00:00
Tobias Guggenmos ac3932ea35 Improve PromQL parser performance by making it non-concurrent (#6356)
Before this commit, the PromQL parser ran in two goroutines:
* The lexer goroutine that splits the input into tokens and sent them over a channel to
* the parser goroutine which produces the abstract syntax tree

The Problem with this approach is that the parser spends more time on goroutine creation
and syncronisation than on actual parsing.

This commit removes that concurrency and replaces the channel by a slice based buffer.

Benchmarks show that this makes the up to 7 times faster than before.

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2019-11-21 18:43:09 +00:00
Harkishen Singh 37d666949c fixes space issue in duration range promql (#6295)
* fix space issue in duration range promql

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>

* updated logic

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>

* fixed lexer to skip over the spaces

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>

* added unittests for updated lexer

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>

* added unittests for updated lexer

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>
2019-11-11 09:56:24 +01:00
Alex Dzyoba e3882629ba promql: Move tests to testutil (#6103)
* promql: Move tests to testutil

Signed-off-by: Alex Dzyoba <alex@dzyoba.com>

* promql: Match error type via errors.As in tests

Signed-off-by: Alex Dzyoba <alex@dzyoba.com>

* promql: Remove unused `expectedList` func from lex_test.go

Signed-off-by: Alex Dzyoba <alex@dzyoba.com>
2019-10-09 18:06:53 -06:00
Julius Volz 8155cc4992
Expose lexer item types (#5358)
* Expose lexer item types

We have generally agreed to expose AST types / values that are necessary
to make sense of the AST outside of the promql package. Currently the
`UnaryExpr`, `BinaryExpr`, and `AggregateExpr` AST nodes store the lexer
item type to indicate the operator type, but since the individual item
types aren't exposed, an external user of the package cannot determine
the operator type. So this PR exposes them.

Although not all item types are required to make sense of the AST (some
are really only used in the lexer), I decided to expose them all here to
be somewhat more consistent. Another option would be to not use lexer
item types at all in AST nodes.

The concrete motivation is my work on the PromQL->Flux transpiler, but
this ought to be useful for other cases as well.

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fix item type names in tests

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-03-14 20:53:55 +01:00
Matt Layher f62fd2bfc9 promql: use subtests in TestLexer
Signed-off-by: Matt Layher <mdlayher@gmail.com>
2019-01-16 16:39:32 -05:00
Ganesh Vernekar dbe55c1352 Subquery (#4831)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-12-22 13:47:13 +00:00
Frederic Branczyk bda9781ccd
Merge pull request #3839 from brancz/remove-old-alert-record
promql: Remove old and unused alerting/reconding syntax
2018-11-06 15:53:27 +01:00
Ganesh Vernekar 73db8b8cea [bugfix] Parse negative value in PromQL (#4564)
* Parse negative value in PromQL
* Enforce space between values

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-13 09:08:01 +01:00
Frederic Branczyk b0b3e3dd74
promql: Remove old and unused alerting/reconding syntax
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
2018-08-07 15:14:06 +02:00
Brian Brazil 99905f82a6 Remove keep_common modifier.
See #3060
2017-10-05 13:27:48 +01:00
Brian Brazil 6f5d952132 Check for invalid utf-8 in lexer strings.
This protects against invalid utf-8 sneaking
in via label_replace.
2017-06-16 15:19:24 +01:00
Alexey Miroshkin bf0e441576 Instantiate lexer inline for the test
Don't use the lex constructor, remove the constructor introduced in the
prevous commit.
2016-08-29 09:20:43 +02:00
Alexey Miroshkin 485f7dde08 Fix data race in lexer and lexer test
As described in #1898 'go test -race' detects a race in lexer code. This
pacth fixes it and also add '-race' option to test target to prevent
regression.
2016-08-26 17:07:17 +02:00
royels 2fdc5717a3 promql: add power binary operation 2016-06-22 23:34:46 -04:00
Ali Reza e7eba75690 remove keeping_extra because it's replaced with keep_common
change all keepExtra label into keepCommon, and move action into removed list

change incorrect token list
2016-05-27 00:02:04 +07:00
Brian Brazil 1d08c4fef0 Add 'ignoring' as modifier for binops.
Where 'on' uses the given labels to match,
'ignoring' uses all other labels to match.

group_left/right is not supported yet.
2016-04-21 19:34:29 +01:00
Tobias Schmidt 8cc86f25c0 Implement relative complement set operator "unless"
The `unless` set operator can be used to return all vector elements from
the LHS which do not match the elements on the RHS. A use case is to
return all metrics for nodes which do not have a specific role:

    node_load1 unless on(instance) chef_role{role="app"}
2016-04-04 01:29:44 -04:00
Fabian Reinartz ab3d7a0ec0 Remove old alerting syntax 2016-03-23 10:19:00 +01:00
Julius Volz 9ea2465b99 Fix typo in lexer test. 2016-03-02 01:13:27 +01:00
Tobias Schmidt 907b1380a7 Add tests to specify the string escaping behavior 2016-03-01 17:23:18 -05:00
Brian Brazil 9d0112d7cf Add without aggregator modifier.
This has the advantage that the user doesn't need
to list all labels they want to keep (as with "by")
but without having to worry about inconsistent labels
as when there's only one time series (as with "keeping_common").

Almost all aggregation should use this rather than the existing
two options as it's much less error prone and easier to maintain
due to not having to always add in "job" plus whatever other common
job-level labels you have like "region".
2016-02-08 14:05:33 +00:00
Fabian Reinartz 6b4a6962d2 Support old alerting rule syntax 2016-01-11 12:14:06 +01:00
Fabian Reinartz 4209ec6864 Change WITH keyword to LABELS 2015-12-23 14:54:02 +01:00
Fabian Reinartz af3a6661ed Implement new alerting rule syntax 2015-12-11 17:02:34 +01:00
Brian Brazil 29e8dc2c49 promql: Add 'bool' modifier to comparison functions
When doing comparison operations on vectors, filtering
sometimes gets in the way and you have to go to a fair bit of
effort to workaround it in order to always return a result.
The 'bool' modifier instead of filtering returns 0/1 depending
on the result of the compairson.

This is also a prerequisite to removing plain scalar/scalar comparisons,
as it maintains the current behaviour under a new syntax.
2015-09-02 14:51:44 +01:00
Julius Volz 4e4b468fba Fix lexer bug treating non-Latin Unicode digits as digits.
Fixes https://github.com/prometheus/prometheus/issues/939
2015-07-29 02:11:13 +02:00
Fabian Reinartz 749ae450c5 promql: add runbook to alert statement.
This commit adds the RUNBOOK keyword to alert statements. The field
is optional and expected to be a link.
2015-06-25 13:00:52 +02:00
Julius Volz 5e2d1c1464 Deprecate keeping_extra, rename it to keep_common.
`keep_common` is more in line with the function name
`drop_common_labels()` terminology-wise, and also more in line with
`group_left`/`group_right` (no `...ing` verb suffix).

We could also go the full way and call it `keep_common_labels`. That
would have the benefit of being even more consistent with the function
`drop_common_labels()` and would be more explanatory, but it also seems
quite long.
2015-06-12 14:21:05 +02:00
Fabian Reinartz c60e3387ef promql: fix lexing of \r as whitespaces 2015-06-02 18:33:49 +02:00
Fabian Reinartz a236c01457 Add time series description parsing.
This commit adds parsing of time series description to the exisiting
query language parser. Time series descriptions are defined by a
metric followed by a sequence of values.
2015-05-18 17:29:32 +02:00
Fabian Reinartz d122749b39 Enhance lexer testing output 2015-05-18 17:19:17 +02:00
Fabian Reinartz 969c231191 Make parser more strict about identifiers, extract number parsing 2015-05-11 11:45:23 +02:00
Fabian Reinartz 8707c54508 Fix single quote parsing, add tests 2015-05-08 16:43:02 +02:00
Fabian Reinartz 279831cdf1 Fix and improve parsing error output. 2015-04-30 12:19:39 +02:00
Fabian Reinartz 25cdff3527 Remove name arg from Parse* functions, enhance parsing errors. 2015-04-29 16:38:41 +02:00
Fabian Reinartz 32b7595c47 Create promql package with lexer/parser.
This commit creates a (so far unused) package. It contains the a custom
lexer/parser for the query language.

ast.go: New AST that interacts well with the parser.
lex.go: Custom lexer (new).
lex_test.go: Lexer tests (new).
parse.go: Custom parser (new).
parse_test.go: Parser tests (new).
functions.go: Changed function type, dummies for parser testing (barely changed/dummies).
printer.go: Adapted from rules/ and adjusted to new AST (mostly unchanged, few additions).
2015-04-23 16:04:50 +02:00