Commit graph

16 commits

Author SHA1 Message Date
Brian Brazil c66aeb3fff
In histogram_quantile merge buckets with equivalent le values (#5158)
This makes things generally more resilient, and will
help with OpenMetrics transitions (and inconsistencies).

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-02-01 10:22:44 +00:00
Mario Trangoni 0e2aa35771 promql: fix unconvert issues (#4040)
See,
$ gometalinter --vendor --disable-all --enable=unconvert --deadline 6m ./...
promql/engine.go:1396:26⚠️ unnecessary conversion (unconvert)
promql/engine.go:1396:40⚠️ unnecessary conversion (unconvert)
promql/engine.go:1398:26⚠️ unnecessary conversion (unconvert)
promql/engine.go:1398:40⚠️ unnecessary conversion (unconvert)
promql/engine.go:1427:26⚠️ unnecessary conversion (unconvert)
promql/engine.go:1427:40⚠️ unnecessary conversion (unconvert)
promql/engine.go:1429:26⚠️ unnecessary conversion (unconvert)
promql/engine.go:1429:40⚠️ unnecessary conversion (unconvert)
promql/engine.go:1505:50⚠️ unnecessary conversion (unconvert)
promql/engine.go:1573:46⚠️ unnecessary conversion (unconvert)
promql/engine.go:1578:46⚠️ unnecessary conversion (unconvert)
promql/engine.go:1591:80⚠️ unnecessary conversion (unconvert)
promql/engine.go:1602:94⚠️ unnecessary conversion (unconvert)
promql/engine.go:1630:18⚠️ unnecessary conversion (unconvert)
promql/engine.go:1631:24⚠️ unnecessary conversion (unconvert)
promql/engine.go:1634:18⚠️ unnecessary conversion (unconvert)
promql/engine.go:1635:34⚠️ unnecessary conversion (unconvert)
promql/functions.go:302:42⚠️ unnecessary conversion (unconvert)
promql/functions.go:315:42⚠️ unnecessary conversion (unconvert)
promql/functions.go:334:26⚠️ unnecessary conversion (unconvert)
promql/functions.go:395:31⚠️ unnecessary conversion (unconvert)
promql/functions.go:406:31⚠️ unnecessary conversion (unconvert)
promql/functions.go:454:27⚠️ unnecessary conversion (unconvert)
promql/functions.go:701:46⚠️ unnecessary conversion (unconvert)
promql/functions.go:701:78⚠️ unnecessary conversion (unconvert)
promql/functions.go:730:43⚠️ unnecessary conversion (unconvert)
promql/functions.go:1220:23⚠️ unnecessary conversion (unconvert)
promql/functions.go:1249:23⚠️ unnecessary conversion (unconvert)
promql/quantile.go:107:54⚠️ unnecessary conversion (unconvert)
promql/quantile.go:182:16⚠️ unnecessary conversion (unconvert)
promql/quantile.go:182:64⚠️ unnecessary conversion (unconvert)

Signed-off-by: Mario Trangoni <mjtrangoni@gmail.com>
2018-06-06 18:20:38 +01:00
Fabian Reinartz 73b8ff0ddc Merge branch 'master' into dev-2.0 2017-04-27 10:19:55 +02:00
Jack Neely 896f951e68 Force buckets in a histogram to be monotonic for quantile estimation (#2610)
* Force buckets in a histogram to be monotonic for quantile estimation

The assumption that bucket counts increase monotonically with increasing
upperBound may be violated during:

  * Recording rule evaluation of histogram_quantile, especially when rate()
     has been applied to the underlying bucket timeseries.
  * Evaluation of histogram_quantile computed over federated bucket
     timeseries, especially when rate() has been applied

This is because scraped data is not made available to RR evalution or
federation atomically, so some buckets are computed with data from the N
most recent scrapes, but the other buckets are missing the most recent
observations.

Monotonicity is usually guaranteed because if a bucket with upper bound
u1 has count c1, then any bucket with a higher upper bound u > u1 must
have counted all c1 observations and perhaps more, so that c  >= c1.

Randomly interspersed partial sampling breaks that guarantee, and rate()
exacerbates it. Specifically, suppose bucket le=1000 has a count of 10 from
4 samples but the bucket with le=2000 has a count of 7, from 3 samples. The
monotonicity is broken. It is exacerbated by rate() because under normal
operation, cumulative counting of buckets will cause the bucket counts to
diverge such that small differences from missing samples are not a problem.
rate() removes this divergence.)

bucketQuantile depends on that monotonicity to do a binary search for the
bucket with the qth percentile count, so breaking the monotonicity
guarantee causes bucketQuantile() to return undefined (nonsense) results.

As a somewhat hacky solution until the Prometheus project is ready to
accept the changes required to make scrapes atomic, we calculate the
"envelope" of the histogram buckets, essentially removing any decreases
in the count between successive buckets.

* Fix up comment docs for ensureMonotonic

* ensureMonotonic: Use switch statement

Use switch statement rather than if/else for better readability.
Process the most frequent cases first.
2017-04-14 16:21:49 +02:00
Fabian Reinartz 9ea10d5265 promql: use labels.Builder to modify labels 2016-12-24 14:35:24 +01:00
Fabian Reinartz c6cd998905 promql: use local labels, add conversion 2016-12-24 14:01:37 +01:00
Fabian Reinartz ff504af2aa promql: undo accidental exports 2016-12-24 11:41:37 +01:00
Fabian Reinartz ac5d3bc05e promql: scalar T/V and Point 2016-12-24 11:23:06 +01:00
Fabian Reinartz a62df87022 promql: rename vector 2016-12-24 10:40:09 +01:00
Fabian Reinartz 15a931dbdb promql: migrate model types, use tsdb interfaces 2016-12-24 00:39:52 +01:00
Brian Brazil 0303ccc6a7 Add quantile aggregator. 2016-07-21 00:09:19 +01:00
Brian Brazil 15f9fe0a45 Factor out quantile fucntion. 2016-07-20 23:56:18 +01:00
Fabian Reinartz d6b8da8d43 Switch promql types to common/model 2015-08-25 13:49:14 +02:00
Fabian Reinartz 306e8468a0 Switch from client_golang/model to common/model 2015-08-21 13:33:38 +02:00
Brian Brazil f34de493d5 Add increase() function, to replace delta(..., 1).
This calculates how much a counter increases over
a given period of time, which is the area under the curve
of it's rate.

increase(x[5m]) is equivilent to rate(x[5m]) * 300.
2015-05-26 22:49:21 +01:00
Fabian Reinartz 5602328c7c Refactor query evaluation.
This copies the evaluation logic from the current rules/ package.
The new engine handles the execution process from query string to final result.
It provides query timeout and cancellation and general flexibility for
future changes.

functions.go: Add evaluation implementation. Slight changes to in/out data but
	not to the processing logic.
quantile.go: No changes.
analyzer.go: No changes.
engine.go: Actually new part. Mainly consists of evaluation methods
	which were not changed.
setup_test.go: Copy of rules/helpers_test.go to setup test storage.
promql_test.go: Copy of rules/rules_test.go.
2015-04-28 14:19:05 +02:00