Commit graph

7834 commits

Author SHA1 Message Date
Callum Styan e87449b59d Remote Write: Queue Manager specific metrics shouldn't exist if the queue no longer exists (#5445)
* Unregister remote write queue manager specific metrics when stopping the
queue manager.

* Use DeleteLabelValues instead of Unregister to remove queue and watcher
related metrics when we stop them. Create those metrics in the structs
start functions rather than in their constructors because of the
ordering of creation, start, and stop in remote storage ApplyConfig.

* Add setMetrics function to WAL watcher so we can set
the watchers metrics in it's Start function, but not
have to call Start in some tests (causes data race).

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-04-23 09:49:17 +01:00
Callum Styan b7538e7b49 Don't stop, recreate, and start remote storage QueueManagers if the (#5485)
remote write config hasn't changed at all.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-04-23 09:47:18 +01:00
Simon Pasquier abc1994bec
cmd/promtool: return errors from rule evaluations (#5483)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-23 09:59:03 +02:00
Bjoern Rabenstein 76102d570c Add test for external labels in label template
Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-21 00:08:39 +02:00
Matthias Loibl 388caa06ac
Use github.com/oklog/run not archived oklog/oklog
Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>
2019-04-19 14:55:28 +02:00
naivewong 7a12d4588a Avoid potential collision of createIdxChkReaders in querier_test.go (#585)
* Make chink ref monotonically increasing number in the `createIdxChkReaders` test func.
Signed-off-by: naivewong <867245430@qq.com>
2019-04-18 16:11:39 +03:00
PrometheusBot e2c6eb2c60 Synchronize Makefile.common from prometheus/prometheus (#584)
Propagating changes from master Makefile.common located in prometheus/prometheus.
2019-04-18 14:26:25 +03:00
Simon Pasquier 7efb8e9480
scripts/sync_makefiles.sh: redirect git-push to /dev/null (#5478)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-18 12:01:49 +02:00
Romain Baugue 95193fa027 Exhaust every request body before closing it (#5166) (#5479)
From the documentation:
> The default HTTP client's Transport may not
> reuse HTTP/1.x "keep-alive" TCP connections if the Body is
> not read to completion and closed.

This effectively enable keep-alive for the fixed requests.

Signed-off-by: Romain Baugue <romain.baugue@elwinar.com>
2019-04-18 09:50:37 +01:00
EarthmanT 35be8c9e25 Add azure public ip label (#5475)
* Update Azure SD Config with Public IP label

Signed-off-by: earthmant <trammell@cloudify.co>
2019-04-17 16:05:44 +01:00
Goutham Veeramachaneni 3cc5f9d880 Make sure subquery range is taken into account for selection (#5467)
* Make sure subquery range is taken into account for selection

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2019-04-17 13:52:41 +01:00
Simon Pasquier 0607462008
scripts/sync_makefiles.sh: mask token in logs (#5465)
Circle CI doesn't mask automatically secret environment variables in
logs.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-17 14:41:06 +02:00
Brian Brazil deeb79fec0
Merge pull request #5470 from prometheus/mergeback291
Merge 2.9.1 release back into master
2019-04-17 09:28:07 +01:00
Björn Rabenstein 81acc1dc0f Update RELEASE.md to reflect mailing list change (#5472)
We are not using `prometheus-users@googlegroups.com` for announcments
anymore.

Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-17 09:27:15 +01:00
Bjoern Rabenstein 38d518c0fe Rework #5009 after comments
Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-17 01:40:10 +02:00
Bjoern Rabenstein a92ef68dd8 Fix staticcheck errors
Not sure why they only show up now.

Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-17 01:40:10 +02:00
Sylvain Rabot 335a34486e Add external labels to template expansion
This affects the expansion of templates in alert labels and
annotations and console templates.

Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>
2019-04-17 01:40:10 +02:00
Vandewilly 1d4c9efe63 Update Go version in README.md (#5471)
Prometheus 2.9 requires Go 1.12+ to build it from source
This commit updates the README file according to this requirement

Signed-off-by: Vandewilly Silva <vandewilly.oli.silva@hpe.com>
2019-04-16 23:33:08 +01:00
Brian Brazil ad71f2785f
Merge pull request #5464 from prometheus/rel291
Prepare 2.9.1
2019-04-16 14:52:27 +01:00
Brian Brazil a9309a2e1e Prepare 2.9.1
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-16 12:54:34 +01:00
Simon Pasquier b7169df0c1 discovery/kubernetes: fix missing label sanitization (#5462)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-16 12:52:47 +01:00
Vasily Sliouniaev ef19ede14a Prevent reshard concurrent with calling stop (#5460)
* Prevent reshard concurrent with calling stop

Signed-off-by: Vasily <v.sliouniaev@gmail.com>
2019-04-16 12:52:47 +01:00
Hrishikesh Barman 27404e6226 better readability for data directory tree in storage docs (#5128)
* better readabilty for data dir tree in storage docs

Signed-off-by: geekodour <hrishikeshbman@gmail.com>
2019-04-16 11:40:13 +01:00
Vasily Sliouniaev 5be9a1426f Prevent reshard concurrent with calling stop (#5460)
* Prevent reshard concurrent with calling stop

Signed-off-by: Vasily <v.sliouniaev@gmail.com>
2019-04-16 11:25:19 +01:00
Simon Pasquier 559237cc4f discovery/kubernetes: fix missing label sanitization (#5462)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-15 19:04:50 +01:00
Johannes Würbach 8dfd829fef Docker images for ARM32v7 and ARM64v8 (#5031)
Build and publish ARM32v7 and ARM64v8 docker images.

Signed-off-by: Johannes Würbach <johannes.wuerbach@googlemail.com>
2019-04-15 14:27:00 +02:00
Brian Brazil 1a9cdbd024
Release 2.9.0 (#5458)
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-15 10:05:07 +01:00
Paweł Krupa 4d7417f5d4 *: add CI job to synchronize Makefile.common across all repos (#5444)
Signed-off-by: paulfantom <pawel@krupa.net.pl>
2019-04-11 14:31:04 +02:00
Julius Volz bc1c7f1809 Fix scalar-vector comparisons (#5454)
* Fix scalar-vector comparisons

Fixes https://github.com/prometheus/prometheus/issues/5452

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-04-11 10:42:16 +01:00
Brian Brazil 46660a0745
Prepare 2.9.0-rc.0 release (#5450)
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-10 18:59:25 +01:00
Brian Brazil 8ff6938fa4
Update dependencies. (#5449)
Including going to tsdb 0.7.0.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-10 18:47:25 +01:00
Bryan Boreham 69dd090880 Check for cancellation on every step of a range evaluation
Signed-off-by: Bryan Boreham <bryan@weave.works>
2019-04-10 13:27:45 +01:00
Bryan Boreham e4a37d0986 Replace select with simpler error check
The documentation for Context states that this is just as good:
	// If Done is not yet closed, Err returns nil.
	// If Done is closed, Err returns a non-nil error

Signed-off-by: Bryan Boreham <bryan@weave.works>
2019-04-10 13:27:45 +01:00
qinng cc75c27580 Fix multiple response.WriteHeader calls error in remote read adapter (#5159)
* fix multiple response.WriteHeader calls in remote read adapter
* remove useless return

Signed-off-by: qinng <guoruyi1@xiaomi.com>
2019-04-10 13:25:35 +01:00
Simon Pasquier e5dbac7972 cmd/prometheus: group flags properly (#5419)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-10 13:22:05 +01:00
Simon Pasquier dafd1632a2 discovery/kubernetes: add present labels for labels/annotations (#5443)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-10 13:21:42 +01:00
Simon Pasquier c1682adb2f Bump prometheus/common to v0.3.0 (#5344)
* Reload certificates from disk automatically

This change bumps github.com/prometheus/common to include
https://github.com/prometheus/common/pull/173

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* scrape: close idle connections on reload/stop

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* use v0.3.0 tag

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-10 13:20:00 +01:00
Krasi Georgiev 4b3a5ac5d3
release 0.7.0 (#580)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-04-10 14:58:20 +03:00
Lucas Servén Marín 0e42449b78 docs/querying: improve wording (#5442)
This commit improves the wording of the subquery examples and makes
them more consistent with other examples.

Signed-off-by: Lucas Servén Marín <lserven@gmail.com>
2019-04-09 17:48:41 +01:00
Frederic Branczyk 141a165eb1
Merge pull request #5441 from prometheus/29docs
Put myself down for 2.9 release.
2019-04-09 17:00:11 +02:00
Brian Brazil c8ee34d0b4 Put myself down for 2.9 release.
Update docs on where to email release announcements.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-09 14:21:24 +01:00
Brian Brazil dfed85e4a4
Keep series that are still in WAL in checkpoints (#577)
If all the samples are deleted for a series,
we should still keep the series in the WAL as
anything else reading the WAL will still care
about it in order to understand the samples.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-09 14:16:24 +01:00
Brian Brazil 259847a6b1
Be smarter in how we look at matchers. (#572)
* Add unittests for PostingsForMatcher.

* Selector methods are all stateless, don't need a reference.

* Be smarter in how we look at matchers.

Look at all matchers to see if a label can be empty.

Optimise Not handling, so i!="2" is a simple lookup
rather than an inverse postings list.

All all the Withouts together, rather than
having to subtract each from all postings.

Change the pre-expand the postings logic to always do it before doing a
Without only. Don't do that if it's already a list.

The initial goal here was that the oft-seen pattern
i=~"something.+",i!="foo",i!="bar" becomes more efficient.

benchmark                                                            old ns/op     new ns/op     delta
BenchmarkHeadPostingForMatchers/n="1"-4                              5888          6160          +4.62%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4                      7190          6640          -7.65%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4                      6038          5923          -1.90%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4                     6030884       4850525       -19.57%
BenchmarkHeadPostingForMatchers/i=~".*"-4                            887377940     230329137     -74.04%
BenchmarkHeadPostingForMatchers/i=~".+"-4                            490316101     319931758     -34.75%
BenchmarkHeadPostingForMatchers/i=~""-4                              594961991     130279313     -78.10%
BenchmarkHeadPostingForMatchers/i!=""-4                              537542388     318751015     -40.70%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4              10460243      8565195       -18.12%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4       44964267      8561546       -80.96%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4                42244885      29137737      -31.03%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4              35285834      32774584      -7.12%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4             8951047       8379024       -6.39%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4       63813335      30672688      -51.93%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4     45381112      44924397      -1.01%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-09 11:59:45 +01:00
Callum Styan c2b88992a3 Remote Write: fix checkpoint reading (#5429)
* Fix ReadCheckpoint to ensure that it actually reads all the contents of
each segment in a checkpoint dir, or returns an error.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-04-09 10:52:44 +01:00
naivewong 7ab060c864 selectOverlappingDirs selects wrong dirs where there are 2 disjoint sets of overlapping dirs
Signed-off-by: naivewong <867245430@qq.com>
2019-04-08 15:27:06 +03:00
zhulongcheng 8b33ee9e2b force persisting the tombstone file (#578)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-04-08 15:06:40 +03:00
Brian Brazil f7184978f4 Protect against memory exhaustion when scraping.
Now that we're not losing the scrape cache across failed
scrape, a scrape that continually failed but had varying
series or metadata (e.g. timestamps in metric names,
plus hitting smaple_limit) would grow the cache indefinitely.

Add some code to catch that, and flush the cache anyway.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-04 19:09:11 +01:00
Brian Brazil dd3073616c Don't lose the scrape cache on a failed scrape.
This avoids CPU usage increasing when the target comes back.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-04 19:09:11 +01:00
Simon Pasquier 4f47806a7d
discovery/dns: fix slice with wrong length (#5432)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-04 17:05:35 +02:00
Simon Pasquier 81c4248081
*: bump gRPC and protobuf dependencies (#5367)
The goal is to remove almost all references to the
golang.org/x/net/context package.

github.com/gogo/protobuf => v1.2.1
google.golang.org/grpc => v1.19.1
github.com/grpc-ecosystem/grpc-gateway => v1.18.5

It also replaces github.com/cockroachdb/cmux by github.com/soheilhy/cmux
because of [1] which fixes #3909 incidentally.

[1] https://github.com/grpc/grpc-go/issues/2636

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-04 11:55:32 +02:00