Commit graph

5786 commits

Author SHA1 Message Date
Tom Wilkie 2fa93595d6
More WAL remote_write tweaks. (#5300)
* Consistently pre-lookup the metrics for a given queue in queue manager.
* Don't open the WAL (for writing) in the remote_write code.
* Add some more logging.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-03-05 12:21:11 +00:00
Krasi Georgiev 1684dc750a
updated tsdb to 0.6.0 (#5292)
* updated tsdb to 0.6.0

as part of the update also added the new storage.tsdb.allow-overlapping-blocks flag and mark it as experimental.
2019-03-04 21:42:45 +02:00
Tariq Ibrahim 1adb91738d fix typo in recordType method of wal_watcher.go (#5297)
Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-04 17:33:35 +01:00
Ganesh Vernekar cfb9135a41
Merge pull request #5290 from prometheus/scalar-crash
Fix panic when aggregator param is not a literal.
2019-03-04 21:51:14 +05:30
Brian Brazil 858c363e94 Fix panic when aggregator param is not a literal.
The return value for checkForSeriesSetExpansion
is always nil, simplify.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-03-04 12:00:05 +00:00
Björn Rabenstein 32130cbbfa
Merge pull request #5285 from prometheus/beorn7/cleanup
Cleanup .gitignore
2019-03-04 11:53:33 +01:00
Tariq Ibrahim 197e5ac597 docs: minor improvements to the service discovery README.md (#5296)
i) Increased the size of the Service Discovery Readme title
ii) Changed `TargetGroups` to "target groups" as it has been relocated and renamed to another package.

Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-03 19:48:03 +01:00
Tariq Ibrahim ab8e9b7423 fix typo in queue_manager.go comment (#5294)
Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-03 11:35:29 +00:00
Goutham Veeramachaneni 82f98c825a
Merge pull request #5291 from gouthamve/sevenpointtwo
*: cut 2.7.2
2019-03-02 06:30:42 -08:00
Goutham Veeramachaneni 535e631621
*: cut 2.7.2
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2019-03-02 15:19:55 +01:00
Tom Wilkie 67da8e7b46
Refactor and fix queue resharding (#5286)
- Remove prometheus_remote_queue_last_send_timestamp_seconds metric.  Its not particularly useful, we have highest_timestamp_seconds.
- Factor out maxGauage, a gauge that only increases.
- Change sharding calculations to use max samples in timestamp - max samples out timestamp (not rates).
- Also include the ratio of samples dropped to correctly predict number of pending samples.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-03-01 11:04:26 -08:00
Paul Gier d8c06bb2b7 Makefile.common: update promu to v0.3.0 (#5280)
Signed-off-by: Paul Gier <pgier@redhat.com>
2019-02-28 19:00:49 +01:00
Callum Styan b8106dd459 Review feedback:
- Add a dropped samples EWMA and use it in calculating desired shards.
- Update metric names and a log messages.
- Limit number of entries in the dedupe logging middleware to prevent potential OOM.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Callum Styan 512f549064 Refactor: inline decodeRecord in readSegment and don't bother decoding samples records if we're not tailing the segment, add a benchmark test and fix some other tests
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie f795942572 Decrement pending sample when queue exits.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie ee7efa93fe Fix some tests.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Callum Styan b69bdfb4d1 Store the checkpoint we read last, so that we don't keep reading the same checkpoint on each tick.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie efbd9559f4 Deal with corruptions in the WAL:
- If we're replaying the WAL to get series records, skip that segment when we hit corruptions.
- If we're tailing the WAL for samples, fail the watcher.
- When the watcher fails, restart from the latest checkpoint - and only send new samples by updating startTime.
- Tidy up log lines and error handling, don't return so many errors on quiting.
- Expect EOF when processing checkpoints.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie 92fcf375b0 Update vendored TSDB version.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie adf5307470 Update wal LiveReader to ensure EOF is correctly propagated.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Callum Styan d6258aea8f Fix up remote write tests:
- Tests that created a QueueManager were leaving behind files at the end of tests.
- WAL replaying (readToEnd)tests seem to require extra time to finish now.
- Some fixes to make staticcheck happy

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie 184f06a981 Combine the record decoding metrics into one; break out garbage collection into a separate function.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie 859cda27ff Remove some 'global' state, moving segment numbers to parameters.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie bdc6b764b0 If reading the WAL fails, try again. Also, read from the segment containing the index for the last checkpoint, not the first segment.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie d6f911b511 Factor out logging ratelimit & dedupe middleware.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie a5c20642b3 Refactor WAL watcher to remove some duplication.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
Tom Wilkie 37ad4db485 Export timestamps in seconds since epoch.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-28 08:38:39 -08:00
beorn7 6e68867edf Cleanup .gitignore
`*-stamp`, `/.build`, `/.release`, `/.tarballs` are remnants from our
old build system.

`*.rules` are Prom1.x rules files.

CircleCI config is now in its own directory.

Signed-off-by: beorn7 <beorn@soundcloud.com>
2019-02-28 16:37:50 +01:00
LongKB 84df210c41 Update prometheus.io's URL to the latest version (#5270)
Currently, the lastest version is **2.7**. But the version in web page is **2.0**.
So this commit aims to update the URL to the latest version of **prometheus.io**

Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>
2019-02-27 10:39:50 +00:00
JoeWrightss e4b88704a6 Fix misspell in manager_test.go (#5279)
Signed-off-by: zhoulin xie <zhoulin.xie@daocloud.io>
2019-02-27 11:22:31 +01:00
Simon Pasquier 1d2fc95b1c
discovery/marathon: pass context to the client (#5232)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-26 14:49:16 +01:00
Simon Pasquier e60d314f43
discovery/consul: pass current context to Consul queries (#5230)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-26 14:48:19 +01:00
Simon Pasquier 8f578d9c6b
discovery/ec2: pass context to the client (#5234)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-26 14:48:03 +01:00
Simon Pasquier 4997dcb4a1
discovery/gce: pass context to the client (#5233)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-26 14:47:43 +01:00
Simon Pasquier 9040dddd0c
discovery/azure: pass context to the client (#5255)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-26 14:47:26 +01:00
Simon Pasquier fe7a1bcfc6
discovery/triton: pass context to the client (#5235)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-26 14:47:04 +01:00
tuanvcw 9de0ab3c8a Update remaining deprecated links in docs (#5271)
Signed-off-by: Vu Cong Tuan <tuanvc@vn.fujitsu.com>
2019-02-26 10:16:38 +00:00
Björn Rabenstein ad29221a7b
Merge pull request #5020 from erikh/upgrade-miekg-dns
Upgrade miekg dns
2019-02-25 12:47:32 +01:00
David Symonds 46361a7c85 rules: Fix sorting of result from (*Manager).RuleGroups (#5260)
The previous code was defective in that it never sorted groups within a
file due to doing a multi-key sort incorrectly.

Signed-off-by: David Symonds <dsymonds@gmail.com>
2019-02-23 09:51:44 +01:00
Simon Pasquier e72c875e63
config: fix Kubernetes config with empty API server (#5256)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-22 15:51:47 +01:00
JoeWrightss 362873f72b Fix .Log() error message (#5257)
Signed-off-by: zhoulin xie <zhoulin.xie@daocloud.io>
2019-02-22 14:39:37 +00:00
LongKB e4a741cb7d Replacing 'HTTP' by 'HTTPS' for securing links (#5252)
Currently, when we access the modified pages with **HTTP**, it is
redirected to **HTTPS** automatically. So this commit aims to
replace **HTTP** to **HTTPs** for security.

Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>
2019-02-22 14:33:02 +01:00
LongKB 23480bef43 Remove the duplicated words (#5251)
Although it is spelling mistakes, it might make an affects while reading.

Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>
2019-02-22 14:32:34 +01:00
Nguyen Hai Truong 5fbda4c9d7 Secure http links (#5244)
Fix http link to https link for secure, modify http to https
in the links of project. Have some http links doesn't
redirect into https.

Co-Authored-By: Nguyen Van Trung trungnv@vn.fujitsu.com
Signed-off-by: Nguyen Hai Truong <truongnh@vn.fujitsu.com>
2019-02-21 10:48:47 +01:00
Ganesh Vernekar 1d9e11a390
Merge pull request #5247 from longkb/fix_typo
Trivial fix: Fix some typos in comments
2019-02-21 10:47:07 +05:30
Ganesh Vernekar ded80bf4a5
Merge pull request #5246 from truongnh1992/removing-redundant-words
Remove duplicated words in comments
2019-02-21 10:45:25 +05:30
Kim Bao Long 94f5352951 Trivial fix: Fix some typos in comments
Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>
2019-02-21 09:07:49 +07:00
Nguyen Hai Truong aed9ea144a Remove duplicated words in comments
Although it is spelling mistakes, it might make an affects
while reading.

Co-Authored-By: Kim Bao Long longkb@vn.fujitsu.com
Signed-off-by: Nguyen Hai Truong <truongnh@vn.fujitsu.com>
2019-02-20 17:41:02 -08:00
Simon Pasquier c8a1a5a93c
discovery/kubernetes: fix support for password_file and bearer_token_file (#5211)
* discovery/kubernetes: fix support for password_file

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Create and pass custom RoundTripper to Kubernetes client

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Use inline HTTPClientConfig

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-20 11:22:34 +01:00
Nguyen Van Duc 89d36a4bf6 Change http to https for security links (#5238)
Signed-off-by: vanduc95 <ducnguyenvan.bk@gmail.com>
2019-02-20 09:50:45 +00:00