Commit graph

7026 commits

Author SHA1 Message Date
Brian Brazil f9bc0817a1
Avoid allocation in mergedPostings.Seek. (#586)
See
https://github.com/prometheus/prometheus/issues/5424#issuecomment-485199651

benchmark                                                            old
ns/op      new ns/op       delta
BenchmarkHeadPostingForMatchers/n="1"-4
5266           5267            +0.02%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4
6469           7801            +20.59%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4
4984           4963            -0.42%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4
7137           7527            +5.46%
BenchmarkHeadPostingForMatchers/i=~".*"-4
8055830839     8117651746      +0.77%
BenchmarkHeadPostingForMatchers/i=~".+"-4
9369298293     9742710251      +3.99%
BenchmarkHeadPostingForMatchers/i=~""-4
2363120708     2507789029      +6.12%
BenchmarkHeadPostingForMatchers/i!=""-4
9387069195     11740628557     +25.07%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4
109668312      107644136       -1.85%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4
106022679      105759760       -0.25%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4
330201810      266421024       -19.32%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4
355627801      292774913       -17.67%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4
108132305      106697690       -1.33%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4
358836972      362579998       +1.04%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4
555879392      575858378       +3.59%

benchmark                                                            old
allocs     new allocs     delta
BenchmarkHeadPostingForMatchers/n="1"-4                              8
8              +0.00%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4                      11
11             +0.00%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4                      11
11             +0.00%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4                     17
17             +0.00%
BenchmarkHeadPostingForMatchers/i=~".*"-4                            58
58             +0.00%
BenchmarkHeadPostingForMatchers/i=~".+"-4
100115         100115         +0.00%
BenchmarkHeadPostingForMatchers/i=~""-4
100125         100125         +0.00%
BenchmarkHeadPostingForMatchers/i!=""-4
100112         100112         +0.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4              19
19             +0.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4       22
22             +0.00%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4
100109         100079         -0.03%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4
100110         100079         -0.03%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4
11176          11167          -0.08%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4
100112         100082         -0.03%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4
111307         111277         -0.03%

benchmark                                                            old
bytes     new bytes     delta
BenchmarkHeadPostingForMatchers/n="1"-4
1360          1360          +0.00%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4
1392          1392          +0.00%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4
1392          1392          +0.00%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4
1520          1520          +0.00%
BenchmarkHeadPostingForMatchers/i=~".*"-4
258254040     258254040     +0.00%
BenchmarkHeadPostingForMatchers/i=~".+"-4
281554280     281554280     +0.00%
BenchmarkHeadPostingForMatchers/i=~""-4
241554888     241554888     +0.00%
BenchmarkHeadPostingForMatchers/i!=""-4
281553664     281553664     +0.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4
1607277       1607277       +0.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4
1607389       1607389       +0.00%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4
73076560      24907600      -65.92%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4
73076765      24907723      -65.92%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4
5416925       3794909       -29.94%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4
73076779      24907819      -65.92%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4
99874860      51705900      -48.23%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-24 13:12:58 +01:00
Callum Styan 3639d51eb6 Remote Storage: string interner should not panic in release (#5487)
* Don't panic if we try to release a string that is not in the interner.

* Move seriesMtx locking in QueueManager's StoreSeries function.

This stops us from calling release for strings that aren't interned if
there's a race between reading a checkpoint and storing new series
labels, which could happen during checkpointing or reloading config.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-04-24 10:46:31 +01:00
Dougal Matthews 24efe92593 Update the required Go version to 1.12 (#5498)
See #5455

Signed-off-by: Dougal Matthews <dougal@redhat.com>
2019-04-24 09:30:54 +02:00
Tariq Ibrahim 00036cd1e5
update client-go,api,api-machinery and klog dependencies
Signed-off-by: Tariq Ibrahim <tariq181290@gmail.com>
2019-04-23 10:54:18 -07:00
Callum Styan e87449b59d Remote Write: Queue Manager specific metrics shouldn't exist if the queue no longer exists (#5445)
* Unregister remote write queue manager specific metrics when stopping the
queue manager.

* Use DeleteLabelValues instead of Unregister to remove queue and watcher
related metrics when we stop them. Create those metrics in the structs
start functions rather than in their constructors because of the
ordering of creation, start, and stop in remote storage ApplyConfig.

* Add setMetrics function to WAL watcher so we can set
the watchers metrics in it's Start function, but not
have to call Start in some tests (causes data race).

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-04-23 09:49:17 +01:00
Callum Styan b7538e7b49 Don't stop, recreate, and start remote storage QueueManagers if the (#5485)
remote write config hasn't changed at all.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-04-23 09:47:18 +01:00
Simon Pasquier abc1994bec
cmd/promtool: return errors from rule evaluations (#5483)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-23 09:59:03 +02:00
Bjoern Rabenstein 76102d570c Add test for external labels in label template
Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-21 00:08:39 +02:00
Matthias Loibl 388caa06ac
Use github.com/oklog/run not archived oklog/oklog
Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>
2019-04-19 14:55:28 +02:00
naivewong 7a12d4588a Avoid potential collision of createIdxChkReaders in querier_test.go (#585)
* Make chink ref monotonically increasing number in the `createIdxChkReaders` test func.
Signed-off-by: naivewong <867245430@qq.com>
2019-04-18 16:11:39 +03:00
PrometheusBot e2c6eb2c60 Synchronize Makefile.common from prometheus/prometheus (#584)
Propagating changes from master Makefile.common located in prometheus/prometheus.
2019-04-18 14:26:25 +03:00
Simon Pasquier 7efb8e9480
scripts/sync_makefiles.sh: redirect git-push to /dev/null (#5478)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-18 12:01:49 +02:00
Romain Baugue 95193fa027 Exhaust every request body before closing it (#5166) (#5479)
From the documentation:
> The default HTTP client's Transport may not
> reuse HTTP/1.x "keep-alive" TCP connections if the Body is
> not read to completion and closed.

This effectively enable keep-alive for the fixed requests.

Signed-off-by: Romain Baugue <romain.baugue@elwinar.com>
2019-04-18 09:50:37 +01:00
EarthmanT 35be8c9e25 Add azure public ip label (#5475)
* Update Azure SD Config with Public IP label

Signed-off-by: earthmant <trammell@cloudify.co>
2019-04-17 16:05:44 +01:00
Goutham Veeramachaneni 3cc5f9d880 Make sure subquery range is taken into account for selection (#5467)
* Make sure subquery range is taken into account for selection

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2019-04-17 13:52:41 +01:00
Simon Pasquier 0607462008
scripts/sync_makefiles.sh: mask token in logs (#5465)
Circle CI doesn't mask automatically secret environment variables in
logs.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-17 14:41:06 +02:00
Brian Brazil deeb79fec0
Merge pull request #5470 from prometheus/mergeback291
Merge 2.9.1 release back into master
2019-04-17 09:28:07 +01:00
Björn Rabenstein 81acc1dc0f Update RELEASE.md to reflect mailing list change (#5472)
We are not using `prometheus-users@googlegroups.com` for announcments
anymore.

Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-17 09:27:15 +01:00
Bjoern Rabenstein 38d518c0fe Rework #5009 after comments
Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-17 01:40:10 +02:00
Bjoern Rabenstein a92ef68dd8 Fix staticcheck errors
Not sure why they only show up now.

Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-17 01:40:10 +02:00
Sylvain Rabot 335a34486e Add external labels to template expansion
This affects the expansion of templates in alert labels and
annotations and console templates.

Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>
2019-04-17 01:40:10 +02:00
Vandewilly 1d4c9efe63 Update Go version in README.md (#5471)
Prometheus 2.9 requires Go 1.12+ to build it from source
This commit updates the README file according to this requirement

Signed-off-by: Vandewilly Silva <vandewilly.oli.silva@hpe.com>
2019-04-16 23:33:08 +01:00
Brian Brazil ad71f2785f
Merge pull request #5464 from prometheus/rel291
Prepare 2.9.1
2019-04-16 14:52:27 +01:00
Brian Brazil a9309a2e1e Prepare 2.9.1
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-16 12:54:34 +01:00
Simon Pasquier b7169df0c1 discovery/kubernetes: fix missing label sanitization (#5462)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-16 12:52:47 +01:00
Vasily Sliouniaev ef19ede14a Prevent reshard concurrent with calling stop (#5460)
* Prevent reshard concurrent with calling stop

Signed-off-by: Vasily <v.sliouniaev@gmail.com>
2019-04-16 12:52:47 +01:00
Hrishikesh Barman 27404e6226 better readability for data directory tree in storage docs (#5128)
* better readabilty for data dir tree in storage docs

Signed-off-by: geekodour <hrishikeshbman@gmail.com>
2019-04-16 11:40:13 +01:00
Vasily Sliouniaev 5be9a1426f Prevent reshard concurrent with calling stop (#5460)
* Prevent reshard concurrent with calling stop

Signed-off-by: Vasily <v.sliouniaev@gmail.com>
2019-04-16 11:25:19 +01:00
Simon Pasquier 559237cc4f discovery/kubernetes: fix missing label sanitization (#5462)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-15 19:04:50 +01:00
Johannes Würbach 8dfd829fef Docker images for ARM32v7 and ARM64v8 (#5031)
Build and publish ARM32v7 and ARM64v8 docker images.

Signed-off-by: Johannes Würbach <johannes.wuerbach@googlemail.com>
2019-04-15 14:27:00 +02:00
Brian Brazil 1a9cdbd024
Release 2.9.0 (#5458)
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-15 10:05:07 +01:00
Paweł Krupa 4d7417f5d4 *: add CI job to synchronize Makefile.common across all repos (#5444)
Signed-off-by: paulfantom <pawel@krupa.net.pl>
2019-04-11 14:31:04 +02:00
Julius Volz bc1c7f1809 Fix scalar-vector comparisons (#5454)
* Fix scalar-vector comparisons

Fixes https://github.com/prometheus/prometheus/issues/5452

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-04-11 10:42:16 +01:00
Brian Brazil 46660a0745
Prepare 2.9.0-rc.0 release (#5450)
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-10 18:59:25 +01:00
Brian Brazil 8ff6938fa4
Update dependencies. (#5449)
Including going to tsdb 0.7.0.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-10 18:47:25 +01:00
Bryan Boreham 69dd090880 Check for cancellation on every step of a range evaluation
Signed-off-by: Bryan Boreham <bryan@weave.works>
2019-04-10 13:27:45 +01:00
Bryan Boreham e4a37d0986 Replace select with simpler error check
The documentation for Context states that this is just as good:
	// If Done is not yet closed, Err returns nil.
	// If Done is closed, Err returns a non-nil error

Signed-off-by: Bryan Boreham <bryan@weave.works>
2019-04-10 13:27:45 +01:00
qinng cc75c27580 Fix multiple response.WriteHeader calls error in remote read adapter (#5159)
* fix multiple response.WriteHeader calls in remote read adapter
* remove useless return

Signed-off-by: qinng <guoruyi1@xiaomi.com>
2019-04-10 13:25:35 +01:00
Simon Pasquier e5dbac7972 cmd/prometheus: group flags properly (#5419)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-10 13:22:05 +01:00
Simon Pasquier dafd1632a2 discovery/kubernetes: add present labels for labels/annotations (#5443)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-10 13:21:42 +01:00
Simon Pasquier c1682adb2f Bump prometheus/common to v0.3.0 (#5344)
* Reload certificates from disk automatically

This change bumps github.com/prometheus/common to include
https://github.com/prometheus/common/pull/173

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* scrape: close idle connections on reload/stop

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* use v0.3.0 tag

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-10 13:20:00 +01:00
Krasi Georgiev 4b3a5ac5d3
release 0.7.0 (#580)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-04-10 14:58:20 +03:00
Lucas Servén Marín 0e42449b78 docs/querying: improve wording (#5442)
This commit improves the wording of the subquery examples and makes
them more consistent with other examples.

Signed-off-by: Lucas Servén Marín <lserven@gmail.com>
2019-04-09 17:48:41 +01:00
Frederic Branczyk 141a165eb1
Merge pull request #5441 from prometheus/29docs
Put myself down for 2.9 release.
2019-04-09 17:00:11 +02:00
Brian Brazil c8ee34d0b4 Put myself down for 2.9 release.
Update docs on where to email release announcements.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-09 14:21:24 +01:00
Brian Brazil dfed85e4a4
Keep series that are still in WAL in checkpoints (#577)
If all the samples are deleted for a series,
we should still keep the series in the WAL as
anything else reading the WAL will still care
about it in order to understand the samples.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-09 14:16:24 +01:00
Brian Brazil 259847a6b1
Be smarter in how we look at matchers. (#572)
* Add unittests for PostingsForMatcher.

* Selector methods are all stateless, don't need a reference.

* Be smarter in how we look at matchers.

Look at all matchers to see if a label can be empty.

Optimise Not handling, so i!="2" is a simple lookup
rather than an inverse postings list.

All all the Withouts together, rather than
having to subtract each from all postings.

Change the pre-expand the postings logic to always do it before doing a
Without only. Don't do that if it's already a list.

The initial goal here was that the oft-seen pattern
i=~"something.+",i!="foo",i!="bar" becomes more efficient.

benchmark                                                            old ns/op     new ns/op     delta
BenchmarkHeadPostingForMatchers/n="1"-4                              5888          6160          +4.62%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-4                      7190          6640          -7.65%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-4                      6038          5923          -1.90%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-4                     6030884       4850525       -19.57%
BenchmarkHeadPostingForMatchers/i=~".*"-4                            887377940     230329137     -74.04%
BenchmarkHeadPostingForMatchers/i=~".+"-4                            490316101     319931758     -34.75%
BenchmarkHeadPostingForMatchers/i=~""-4                              594961991     130279313     -78.10%
BenchmarkHeadPostingForMatchers/i!=""-4                              537542388     318751015     -40.70%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-4              10460243      8565195       -18.12%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-4       44964267      8561546       -80.96%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-4                42244885      29137737      -31.03%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-4              35285834      32774584      -7.12%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-4             8951047       8379024       -6.39%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-4       63813335      30672688      -51.93%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-4     45381112      44924397      -1.01%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-04-09 11:59:45 +01:00
Callum Styan c2b88992a3 Remote Write: fix checkpoint reading (#5429)
* Fix ReadCheckpoint to ensure that it actually reads all the contents of
each segment in a checkpoint dir, or returns an error.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-04-09 10:52:44 +01:00
naivewong 7ab060c864 selectOverlappingDirs selects wrong dirs where there are 2 disjoint sets of overlapping dirs
Signed-off-by: naivewong <867245430@qq.com>
2019-04-08 15:27:06 +03:00
zhulongcheng 8b33ee9e2b force persisting the tombstone file (#578)
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-04-08 15:06:40 +03:00