Commit graph

64 commits

Author SHA1 Message Date
Ganesh Vernekar 73db8b8cea [bugfix] Parse negative value in PromQL (#4564)
* Parse negative value in PromQL
* Enforce space between values

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-13 09:08:01 +01:00
Karsten Weiss d79d573f71 Fix spelling mistakes found by codespell (#4065)
Signed-off-by: Karsten Weiss <knweiss@gmail.com>
2018-04-27 13:04:02 +01:00
Warren Fernandes d49a3df55b Parser test cleanup (#3977)
* parser test cleanup

- Test against the exported package functions instead of the private functions.

* Improves readability of TestParseSeries

- Moves package function closer to parser function
2018-03-20 14:30:52 +00:00
Brian Brazil 99905f82a6 Remove keep_common modifier.
See #3060
2017-10-05 13:27:48 +01:00
Fabian Reinartz ca2b68889b Merge branch 'master' into dev-2.0 2017-06-23 13:15:44 +02:00
Fabian Reinartz f46a8e9ea4 Merge pull request #2854 from prometheus/promql-rune
Check for invalid utf-8 in lexer strings.
2017-06-17 14:42:20 +02:00
Brian Brazil 6f5d952132 Check for invalid utf-8 in lexer strings.
This protects against invalid utf-8 sneaking
in via label_replace.
2017-06-16 15:19:24 +01:00
Harsh Agarwal 16867c89a7 implement label_join issue 1147 (#2806)
Replace OptionalArgs int with Variadic int.
2017-06-16 14:51:22 +01:00
Fabian Reinartz 71fe0c58a8 promql: misc fixes 2016-12-28 11:32:15 +01:00
Fabian Reinartz fecf9532b9 *: fix misc compile errors 2016-12-25 11:42:57 +01:00
Fabian Reinartz ac5d3bc05e promql: scalar T/V and Point 2016-12-24 11:23:06 +01:00
Fabian Reinartz a62df87022 promql: rename vector 2016-12-24 10:40:09 +01:00
Fabian Reinartz 15a931dbdb promql: migrate model types, use tsdb interfaces 2016-12-24 00:39:52 +01:00
Tristan Colgate 68fc15fe4e Report type names in the form used in documentation 2016-11-18 10:12:55 +00:00
Tobias Schmidt 29ced0090f Fix common english misspellings 2016-09-14 23:23:28 -04:00
Tobias Schmidt 04ae6196f2 Fix parsing of label names which are also keywords
The current separation between lexer and parser is a bit fuzzy when it
comes to operators, aggregators and other keywords. The lexer already
tries to determine the type of a token, even though that type might
change depending on the context.

This led to the problematic behavior that no tokens known to the lexer
could be used as label names, including operators (and, by, ...),
aggregators (count, quantile, ...) or other keywords (for, offset, ...).

This change additionally checks whether an identifier is one of these
types. We might want to check whether the specific item identification
should be moved from the lexer to the parser.
2016-09-07 17:45:58 -04:00
beorn7 fc6737b7fb storage: improve index lookups
tl;dr: This is not a fundamental solution to the indexing problem
(like tindex is) but it at least avoids utilizing the intersection
problem to the greatest possible amount.

In more detail:

Imagine the following query:

    nicely:aggregating:rule{job="foo",env="prod"}

While it uses a nicely aggregating recording rule (which might have a
very low cardinality), Prometheus still intersects the low number of
fingerprints for `{__name__="nicely:aggregating:rule"}` with the many
thousands of fingerprints matching `{job="foo"}` and with the millions
of fingerprints matching `{env="prod"}`. This totally innocuous query
is dead slow if the Prometheus server has a lot of time series with
the `{env="prod"}` label. Ironically, if you make the query more
complicated, it becomes blazingly fast:

    nicely:aggregating:rule{job=~"foo",env=~"prod"}

Why so? Because Prometheus only intersects with non-Equal matchers if
there are no Equal matchers. That's good in this case because it
retrieves the few fingerprints for
`{__name__="nicely:aggregating:rule"}` and then starts right ahead to
retrieve the metric for those FPs and checking individually if they
match the other matchers.

This change is generalizing the idea of when to stop intersecting FPs
and go into "retrieve metrics and check them individually against
remaining matchers" mode:

- First, sort all matchers by "expected cardinality". Matchers
  matching the empty string are always worst (and never used for
  intersections). Equal matchers are in general consider best, but by
  using some crude heuristics, we declare some better than others
  (instance labels or anything that looks like a recording rule).

- Then go through the matchers until we hit a threshold of remaining
  FPs in the intersection. This threshold is higher if we are already
  in the non-Equal matcher area as intersection is even more expensive
  here.

- Once the threshold has been reached (or we have run out of matchers
  that do not match the empty string), start with "retrieve metrics
  and check them individually against remaining matchers".

A beefy server at SoundCloud was spending 67% of its CPU time in index
lookups (fingerprintsForLabelPairs), serving mostly a dashboard that
is exclusively built with recording rules. With this change, it spends
only 35% in fingerprintsForLabelPairs. The CPU usage dropped from 26
cores to 18 cores. The median latency for query_range dropped from 14s
to 50ms(!). As expected, higher percentile latency didn't improve that
much because the new approach is _occasionally_ running into the worst
case while the old one was _systematically_ doing so. The 99th
percentile latency is now about as high as the median before (14s)
while it was almost twice as high before (26s).
2016-07-20 17:35:53 +02:00
Brian Brazil 16690736ab Add count_values() aggregator.
This is useful for counting how many instances
of a job are running a particular version/build.

Fixes #622
2016-07-05 17:14:01 +01:00
Brian Brazil 7f23a4a099 Add type check on topk/bottomk parameter. 2016-07-04 18:03:05 +01:00
Brian Brazil 3e5136e36d Make topk/bottomk aggregators. 2016-07-04 13:18:19 +01:00
Brian Brazil 3b89616d82 Allow on, ignoring, by and without wit empty laberls.
This offers new semantics in allowing on() for matching
two single-element vectors with no known common labels.
Previosuly this was often done using on(dummy).

This also allows making it explict that you meant
to do an aggregation without labels via by().

Fixes #1597.
2016-06-24 14:12:51 +01:00
Brian Brazil 246a817300 Flip vector matching to be ignoring by default.
This is a noop semantically.
2016-06-23 17:23:44 +01:00
Ali Reza e7eba75690 remove keeping_extra because it's replaced with keep_common
change all keepExtra label into keepCommon, and move action into removed list

change incorrect token list
2016-05-27 00:02:04 +07:00
Brian Brazil 68e70d992a Clarify error message around on(x) group_left(x) 2016-04-26 14:31:00 +01:00
Brian Brazil 7201c010c4 Rename On to MatchingLabels 2016-04-26 14:28:36 +01:00
Brian Brazil d991f0cf47 For many-to-one matches, always copy label from one side.
This is a breaking change for everyone using the machine roles
labeling approach.
2016-04-21 19:35:41 +01:00
Brian Brazil d1edfb25b3 Add support for OneToMany with IGNORING.
The labels listed in the group_ modifier will be copied from the one
side to the many side. It will be valid to specify no labels.

This is intended to replace the existing ON/GROUP_* support.,
2016-04-21 19:35:35 +01:00
Brian Brazil 1d08c4fef0 Add 'ignoring' as modifier for binops.
Where 'on' uses the given labels to match,
'ignoring' uses all other labels to match.

group_left/right is not supported yet.
2016-04-21 19:34:29 +01:00
Tobias Schmidt 8cc86f25c0 Implement relative complement set operator "unless"
The `unless` set operator can be used to return all vector elements from
the LHS which do not match the elements on the RHS. A use case is to
return all metrics for nodes which do not have a specific role:

    node_load1 unless on(instance) chef_role{role="app"}
2016-04-04 01:29:44 -04:00
Fabian Reinartz ab3d7a0ec0 Remove old alerting syntax 2016-03-23 10:19:00 +01:00
Patrick Bogen 2062fbae0f rewrite operator balancing to be recursive 2016-03-02 15:56:40 -08:00
Brian Brazil 9d0112d7cf Add without aggregator modifier.
This has the advantage that the user doesn't need
to list all labels they want to keep (as with "by")
but without having to worry about inconsistent labels
as when there's only one time series (as with "keeping_common").

Almost all aggregation should use this rather than the existing
two options as it's much less error prone and easier to maintain
due to not having to always add in "job" plus whatever other common
job-level labels you have like "region".
2016-02-08 14:05:33 +00:00
Tobias Schmidt 1a91cd6e09 Rename matrix to range selector in external error messages
The documentation speaks about range vectors and range vector selectors.
This change does not fix all issues, we might still expose the term
"Matrix" in error messages using %T.
2016-01-25 13:25:56 -05:00
Tobias Schmidt 411ca4dba1 Consolidate offset modifier parsing
Remove duplicated offset modifier parsing and ensure offset can only
appear at the end of a selector statement.
2016-01-24 23:11:44 -05:00
Fabian Reinartz 6b4a6962d2 Support old alerting rule syntax 2016-01-11 12:14:06 +01:00
Fabian Reinartz 4209ec6864 Change WITH keyword to LABELS 2015-12-23 14:54:02 +01:00
Fabian Reinartz af3a6661ed Implement new alerting rule syntax 2015-12-11 17:02:34 +01:00
Brian Brazil c36961130b promql: Remove scalar/scalar comparisons.
This change is breaking, use the 'bool' modifier for such comprisons.

After this change all comparisons without 'bool' will filter, and all
comparisons with 'bool' will return 0/1. This makes the language more
consistent and orthogonal, and ultimately easier to learn and use.

If we ever figure out sane semantics for filtering scalar/scalar
comparisons we can add them in, which will most likely come out of how
the new vector() function is used.
2015-10-11 08:51:04 +01:00
Julius Volz 46c5260761 Support escape sequences in strings and add raw strings.
This adapts some functionality from the Go standard library for string
literal lexing and unquoting/unescaping.

The following string types are now supported:

Double- or single-quoted strings:

  These support all escape sequences that Go supports in double-quoted
  string literals. The difference is that Prometheus also has
  single-quoted strings (instead of single-quoted runes in Go). Raw
  newlines are not allowed.

Backtick-quoted raw strings:

  Strings quoted in backticks are treated as raw strings just like in Go
  and may contain raw newlines and other special characters directly.

Fixes https://github.com/prometheus/prometheus/issues/1122
Fixes https://github.com/prometheus/prometheus/issues/1121
2015-10-08 19:17:21 +02:00
Fabian Reinartz 171f50706a Fix unkeyed field errors. 2015-09-18 17:00:08 +02:00
Brian Brazil 9ec11b1847 Merge pull request #1049 from prometheus/bool-nofilter
promql: Add 'bool' modifier to comparison functions
2015-09-03 15:08:38 +01:00
Brian Brazil 29e8dc2c49 promql: Add 'bool' modifier to comparison functions
When doing comparison operations on vectors, filtering
sometimes gets in the way and you have to go to a fair bit of
effort to workaround it in order to always return a result.
The 'bool' modifier instead of filtering returns 0/1 depending
on the result of the compairson.

This is also a prerequisite to removing plain scalar/scalar comparisons,
as it maintains the current behaviour under a new syntax.
2015-09-02 14:51:44 +01:00
Julius Volz 963ad82dcb Fix "go vet" errors.
I ignored all errors of the type "composite literal uses unkeyed
fields". Most of them are wrong because of
https://github.com/golang/go/issues/9171.
2015-08-26 02:05:04 +02:00
Fabian Reinartz 438e232c9b Fix grouping of import blocks 2015-08-22 09:42:45 +02:00
Fabian Reinartz 306e8468a0 Switch from client_golang/model to common/model 2015-08-21 13:33:38 +02:00
Brian Brazil 296f551418 Merge pull request #1014 from prometheus/scalar-rules
rules: Allow recorded rules expressions to be scalars.
2015-08-19 22:10:49 +01:00
Brian Brazil e6a67476c2 rules: Allow recorded rules expressions to be scalars.
This is useful if you want to build up a constant metric,
such as a set of alert thresholds that vary by label value.
2015-08-19 21:09:00 +01:00
Laurie Malau cdf38ab93a Log runtime errors during query evaluation instead of panicking. 2015-08-19 16:56:41 +02:00
Fabian Reinartz 579fdf65e2 Implement unary expression for vector types.
Closes #956
2015-08-04 15:46:36 +02:00
Fabian Reinartz adf109795c forbid unexpected (runtime) errors in parse tests 2015-08-03 12:53:31 +02:00