Commit graph

5001 commits

Author SHA1 Message Date
Callum Styan d0ee4da932 fix minor issues in custom SD example (#4278)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2018-06-18 16:08:02 +01:00
Simon Pasquier 6eab4bbca1 kubernetes_sd: fix namespace filtering (#4273)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-15 09:08:14 +01:00
Fabian Reinartz e7cfc7dae5 web: restore old path prefix behavior
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-15 09:08:14 +01:00
Alin Sinpalean 91ce63a140 Log the line when failing a PromQL test. (#4272)
Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>
2018-06-14 15:18:16 +01:00
Paul Gier d24d2acd11 config: set target group source index during unmarshalling (#4245)
* config: set target group source index during unmarshalling

Fixes issue #4214 where the scrape pool is unnecessarily reloaded for a
config reload where the config hasn't changed.  Previously, the discovery
manager changed the static config after loading which caused the in-memory
config to differ from a freshly reloaded config.

Signed-off-by: Paul Gier <pgier@redhat.com>

* [issue #4214] Test that static targets are not modified by discovery manager

Signed-off-by: Paul Gier <pgier@redhat.com>
2018-06-13 16:34:59 +01:00
Fabian Reinartz 7a74689973
Merge pull request #4258 from prometheus/rmsecheaders
web: remove security headers
2018-06-12 16:21:35 +02:00
Fabian Reinartz 8a4bda8d57 web: remove security headers
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-12 08:59:13 -04:00
Simon Pasquier 0e5e7f75cd discovery/file: fix logging (#4178)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-12 12:45:59 +01:00
Henri DF a3ddde0800 Add prompb/README (#4222)
Signed-off-by: Henri DF <henridf@gmail.com>
2018-06-11 22:00:33 +01:00
Simon Pasquier 45b8dea320 Makefile: update .PHONY target (#4234)
Makefile: update .PHONY target

* Move .PHONY declarations near their targets

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-11 17:51:28 +02:00
Brian Brazil 25e2d9f152
Update example console template for node exporter 0.16.0 (#4208)
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-06-08 14:01:05 +01:00
Andreas Auernhammer 37d1bcf495 limit size of POST requests against remote read endpoint (#4239)
This commit fixes a denial-of-service issue of the remote
read endpoint. It limits the size of the POST request body
to 32 MB such that clients cannot write arbitrary amounts
of data to the server memory.

Fixes #4238

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2018-06-08 08:19:20 +01:00
Fabian Reinartz 32223e1dfb
Merge pull request #4237 from prometheus/release-2.3
Release 2.3
2018-06-07 23:55:01 +02:00
Thomas Jackson 404abe0f1c Bubble up errors to promql from populating iterators (#4136)
This changes the Walk/Inspect API inside the promql package to bubble
up errors. This is done by having the inspector return an error (instead
of a bool) and then bubbling that up in the Walk. This way if any error
is encountered in the Walk() the walk will stop and return the error.
This avoids issues where errors from the Querier where being ignored
(causing incorrect promql evaluation).

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>

Fixes #4136
2018-06-07 17:27:34 +01:00
Fabian Reinartz 04b12703f2 Update changelog
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-07 11:23:12 -04:00
Mario Trangoni 0e2aa35771 promql: fix unconvert issues (#4040)
See,
$ gometalinter --vendor --disable-all --enable=unconvert --deadline 6m ./...
promql/engine.go:1396:26⚠️ unnecessary conversion (unconvert)
promql/engine.go:1396:40⚠️ unnecessary conversion (unconvert)
promql/engine.go:1398:26⚠️ unnecessary conversion (unconvert)
promql/engine.go:1398:40⚠️ unnecessary conversion (unconvert)
promql/engine.go:1427:26⚠️ unnecessary conversion (unconvert)
promql/engine.go:1427:40⚠️ unnecessary conversion (unconvert)
promql/engine.go:1429:26⚠️ unnecessary conversion (unconvert)
promql/engine.go:1429:40⚠️ unnecessary conversion (unconvert)
promql/engine.go:1505:50⚠️ unnecessary conversion (unconvert)
promql/engine.go:1573:46⚠️ unnecessary conversion (unconvert)
promql/engine.go:1578:46⚠️ unnecessary conversion (unconvert)
promql/engine.go:1591:80⚠️ unnecessary conversion (unconvert)
promql/engine.go:1602:94⚠️ unnecessary conversion (unconvert)
promql/engine.go:1630:18⚠️ unnecessary conversion (unconvert)
promql/engine.go:1631:24⚠️ unnecessary conversion (unconvert)
promql/engine.go:1634:18⚠️ unnecessary conversion (unconvert)
promql/engine.go:1635:34⚠️ unnecessary conversion (unconvert)
promql/functions.go:302:42⚠️ unnecessary conversion (unconvert)
promql/functions.go:315:42⚠️ unnecessary conversion (unconvert)
promql/functions.go:334:26⚠️ unnecessary conversion (unconvert)
promql/functions.go:395:31⚠️ unnecessary conversion (unconvert)
promql/functions.go:406:31⚠️ unnecessary conversion (unconvert)
promql/functions.go:454:27⚠️ unnecessary conversion (unconvert)
promql/functions.go:701:46⚠️ unnecessary conversion (unconvert)
promql/functions.go:701:78⚠️ unnecessary conversion (unconvert)
promql/functions.go:730:43⚠️ unnecessary conversion (unconvert)
promql/functions.go:1220:23⚠️ unnecessary conversion (unconvert)
promql/functions.go:1249:23⚠️ unnecessary conversion (unconvert)
promql/quantile.go:107:54⚠️ unnecessary conversion (unconvert)
promql/quantile.go:182:16⚠️ unnecessary conversion (unconvert)
promql/quantile.go:182:64⚠️ unnecessary conversion (unconvert)

Signed-off-by: Mario Trangoni <mjtrangoni@gmail.com>
2018-06-06 18:20:38 +01:00
Fabian Reinartz e4bb42208f
Merge pull request #4183 from prometheus/metastore
scrape,api: provide per-target metric metadata
2018-06-06 18:23:53 +02:00
Fabian Reinartz 057a5ae2b1 Address comments
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-06 11:21:17 -04:00
Fabian Reinartz 0ff42e754e pkg/textparse: unescape help string
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-06 05:56:10 -04:00
Fabian Reinartz ad4c33c1ff scrape,api: provide per-target metric metadata
This adds a per-target cache of scraped metadata. The metadata is only
available for the lifecycle of the attached target. An API endpoint allows
to select metadata by metric name and a label selection of targets.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-06 05:56:10 -04:00
Fabian Reinartz 290d71791a *: cut v2.3.0
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-05 07:35:48 -04:00
Fabian Reinartz 0e7112330e
Merge pull request #4162 from prometheus/scrapemeta
pkg/textparse: refactor and add metadata handling
2018-06-05 12:07:49 +02:00
Fabian Reinartz 37dc0adcbd
Merge pull request #4217 from prometheus/update
Update TSDB and merge release-2.2
2018-06-05 11:57:49 +02:00
Fabian Reinartz 32324f79c7 vendour: update prometheus/tsdb
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-05 05:33:04 -04:00
Fabian Reinartz 9c83e815d1 Merge branch 'release-2.2' into update 2018-06-05 05:29:11 -04:00
Simon Pasquier 097439b0f9 web: limit the number of dropped targets (#4212)
Displaying all the dropped targets in the service-discovery page hurts
the Prometheus server as well as the browser when thousands of dropped
targets exist. This change limits this number to 1,000 and display the
number of active/total targets per scrape configuration.

Add warning when more than 100 targets are dropped

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-05 10:20:32 +01:00
Fabian Reinartz 5fcc5b6028 pkg/textparse: support null bytes in comments and help
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-04 11:34:34 -04:00
Fabian Reinartz fe80dddbc4
Merge pull request #4210 from bboreham/log-remote-name
Add queue name to logger for remote writes
2018-06-04 15:49:39 +02:00
Brian Brazil dd6781add2 Optimise PromQL (#3966)
* Move range logic to 'eval'

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make aggregegate range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* PromQL is statically typed, so don't eval to find the type.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Extend rangewrapper to multiple exprs

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Start making function evaluation ranged

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make instant queries a special case of range queries

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Eliminate evalString

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Evaluate range vector functions one series at a time

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make unary operators range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make binops range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Pass time to range-aware functions.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make simple _over_time functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reduce allocs when working with matrix selectors

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add basic benchmark for range evaluation

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse objects for function arguments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Do dropmetricname and allocating output vector only once.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add range-aware support for range vector functions with params

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise holt_winters, cut cpu and allocs by ~25%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make rate&friends range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make more functions range aware. Document calling convention.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make date functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make simple math functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Convert more functions to be range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make more functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Specialcase timestamp() with vector selector arg for range awareness

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove transition code for functions

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove the rest of the engine transition code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove more obselete code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove the last uses of the eval* functions

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove engine finalizers to prevent corruption

The finalizers set by matrixSelector were being called
just before the value they were retruning to the pool
was then being provided to the caller. Thus a concurrent query
could corrupt the data that the user has just been returned.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add new benchmark suite for range functinos

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Migrate existing benchmarks to new system

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Expand promql benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Simply test by removing unused range code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* When testing instant queries, check range queries too.

To protect against subsequent steps in a range query being
affected by the previous steps, add a test that evaluates
an instant query that we know works again as a range query
with the tiimestamp we care about not being the first step.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse ring for matrix iters. Put query results back in pool.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse buffer when iterating over matrix selectors

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Unary minus should remove metric name

Cut down benchmarks for faster runs.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reduce repetition in benchmark test cases

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Work series by series when doing normal vectorSelectors

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise benchmark setup, cuts time by 60%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Have rangeWrapper use an evalNodeHelper to cache across steps

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Use evalNodeHelper with functions

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Cache dropMetricName within a node evaluation.

This saves both the calculations and allocs done by dropMetricName
across steps.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse input vectors in rangewrapper

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse the point slices in the matrixes input/output by rangeWrapper

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make benchmark setup faster using AddFast

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Simplify benchmark code.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add caching in VectorBinop

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Use xor to have one-level resultMetric hash key

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add more benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Call Query.Close in apiv1

This allows point slices allocated for the response data
to be reused by later queries, saving allocations.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise histogram_quantile

It's now 5-10% faster with 97% less garbage generated for 1k steps

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make the input collection in rangeVector linear rather than quadratic

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise label_replace, for 1k steps 15x fewer allocs and 3x faster

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise label_join, 1.8x faster and 11x less memory for 1k steps

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Expand benchmarks, cleanup comments, simplify numSteps logic.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Address Fabian's comments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Comments from Alin.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Address jrv's comments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove dead code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Address Simon's comments.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Rename populateIterators, pre-init some sizes

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Handle case where function has non-matrix args first

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Split rangeWrapper out to rangeEval function, improve comments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Cleanup and make things more consistent

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make EvalNodeHelper public

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Fabian's comments.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-06-04 15:47:45 +02:00
Alin Sinpalean 9dc763cc03 Run rule evaluation with timestamps precisely evaluation_interval apart (#4201)
* Run rule evaluation with timestamps precisely evaluation_interval apart from one another.

Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>
2018-06-01 15:23:07 +01:00
Bryan Boreham 05cf674279 Mention the DCO (#4211)
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2018-06-01 14:15:12 +01:00
Bryan Boreham 3277aeefaa Add queue name to logger for remote writes
More than one remote_write destination can be configured, in which
case it's essential to know which one each log message refers to.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2018-06-01 13:04:00 +00:00
Callum Styan 03578d5df8 add example usage of SD adapter for converting unsupported SD type to filesd (#3720)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2018-05-30 13:14:34 +01:00
Fabian Reinartz d29e0ba16b
Merge pull request #4179 from krasi-georgiev/update-tsdb
update tsdb
2018-05-29 17:35:57 -04:00
Krasi Georgiev 229f4e63f7 update tsdb
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-05-29 20:51:41 +01:00
Tom Wilkie 12a1a22a5a
Merge pull request #4187 from prometheus/2972-remote-block-shutdown
Only give remote queues 1 minute to flush samples on shutdown.
2018-05-29 14:25:19 +01:00
Tom Wilkie b58199bf12 Review feedback.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-05-29 11:35:43 +01:00
Tom Wilkie 3353bbd018 Add proper unclean shutdown handling with a cancellable context.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-05-29 09:51:29 +01:00
Fabian Reinartz 56e6679281
Merge pull request #4174 from adamdecaf/upate-common-2018-05-18
update vendor'd github.com/prometheus/common/config
2018-05-28 15:18:43 -04:00
Fabian Reinartz 76a4a46cb0 pkg/textparse: refactor and add metadata handling
Extends the parser to allow retrieving metadata.
The lexer now yields proper tokens that are fed into a hand-written
parser on top.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-05-25 11:27:25 -04:00
Tom Wilkie 8acad5f3cd make it compile
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-05-24 15:40:24 +01:00
Tom Wilkie e51d6c4b6c Make remote flush deadline a command line param.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-05-23 15:06:01 +01:00
Tom Wilkie a6c353613a Make the flush deadline configurable.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-05-23 15:04:36 +01:00
Tom Wilkie aa17263edd Remove WaitGroup and extra goroutine.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-05-23 15:04:34 +01:00
Tom Wilkie f3c61f8bb2 Only give remote queues 1 minute to flush samples on shutdown.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-05-23 15:04:32 +01:00
Tom Wilkie 18e6fa7c8a
Merge pull request #4185 from prometheus/4184-dedup-read-recent
Dedupe samples in the mergeIterator.
2018-05-23 12:43:07 +01:00
Ben Kochie 2b228b4260
Merge pull request #4182 from prometheus/superq/circle_attach
Fix CircleCI config
2018-05-23 13:36:12 +02:00
Tom Wilkie ba418780be Dedupe samples in the mergeIterator.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-05-23 12:15:47 +01:00
Ben Kochie 1751c527b1
Fix CircleCI config
Re-add workspace attachment removed by #4132.

Signed-off-by: Ben Kochie <superq@gmail.com>
2018-05-22 18:53:36 +02:00
Adam Shannon ea761c5ab9 update vendor'd github.com/prometheus/common
Signed-off-by: Adam Shannon <adamkshannon@gmail.com>
2018-05-22 09:19:11 -05:00