Commit graph

7569 commits

Author SHA1 Message Date
Ben Kochie e4a49f1a71
Update Makefile.common (#4932)
* Fixup PROMU unpack

When setting up promu, use an explicit path, not the `PROMU` variable.
This allows for Makefile override of the PROMU command line flags.

* Use temp dir for unpacking tools.
* Use BSD compatible tar command.
* OpenBSD mkdir doesn't support `-v`.

Signed-off-by: Ben Kochie <superq@gmail.com>
2018-11-30 16:20:50 +01:00
Krasi Georgiev 48efdf8b81
refactor NewSegmentsRangeReader to take multi WAL ranges (#449)
* refactor NewSegmentsRangeReader to take multi WAL ranges

In case of an error when checkpointing the WAL the error doesn't show
the exact WAL index that is corrupter. this is because it uses
MultiReader to read multiply WAL files.
This refactoring allows the NewSegmentsRangeReader to take more than a
single WAL range and it reads all of the ranges by iterating each one.

this changes the logs from
create checkpoint: read segments: corruption after 4841144384 bytes:...
to
create checkpoint: read segments: corruption in segment
data/wal/00017351 at 123142208: ...

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-30 16:46:16 +02:00
mknapphrt f0e9196dca Return warnings on a remote read fail (#4832)
Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>
2018-11-30 14:27:12 +00:00
Simon Pasquier 77ee41084f
web: group targets by job then instance (#4898)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-30 15:24:15 +01:00
Simon Pasquier 632626b553
web: deduplicate handler label for HTTP metrics (#4752)
* web: deduplicate handler label for HTTP metrics

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* remove instrumentHandlerFunc

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-30 13:30:32 +01:00
Krasi Georgiev 0493efb7c5
repair wal when the record cannot be decoded (#453)
* repair wal when the record cannot be decoded

Currently repair is run only when the error happens in the reader.

A corruption can occur after the record is read and when it is decoded.
This change wraps the error at decoding as a CorruptionErr as this error
is expected to trigger a repair.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-30 13:37:04 +02:00
Tariq Ibrahim 61cf4365d6 add logic to check if an azure VM is deallocated or not (#4908)
* add logic to check if an azure VM is deallocated or not
* update documentation  with the new azure power state label

Signed-off-by: tariqibrahim <tariq.ibrahim@microsoft.com>
2018-11-30 11:32:40 +00:00
Serghei Anicheev 8e659a5109 Adding private_dns_name to the list of ec2 labels which can be used i… (#4693)
* Adding private_dns_name to the list of ec2 labels which can be used in node naming for dynamic environments

Signed-off-by: Serghei Anicheev <serghei@rentalcover.com>
2018-11-30 11:11:06 +00:00
Yaqiang Wang 8b85d876f2 fix file_sd never stop update 'custom_sd.json' file in adapter.go (#4567)
Signed-off-by: wangyaqiang1 <wangyaqiang1@jd.com>
2018-11-30 10:32:17 +01:00
Fabian Reinartz 76b266b2f0
Update Fabian's email address 2018-11-30 09:37:40 +01:00
mengnan a5d39361ab discovery/azure: Fail hard when Azure authentication parameters are missing (#4907)
* discovery/azure: fail hard when client_id/client_secret is empty

Signed-off-by: mengnan <supernan1994@gmail.com>

* discovery/azure: fail hard when authentication parameters are missing

Signed-off-by: mengnan <supernan1994@gmail.com>

* add unit test

Signed-off-by: mengnan <supernan1994@gmail.com>

* add unit test

Signed-off-by: mengnan <supernan1994@gmail.com>

* format code

Signed-off-by: mengnan <supernan1994@gmail.com>
2018-11-29 16:47:59 +01:00
tariqibrahim e4222b1223 update client-go to v9.0.0 and other k8s deps to v1.12.3
Signed-off-by: tariqibrahim <tariq.ibrahim@microsoft.com>
2018-11-28 21:13:03 -08:00
Krasi Georgiev 0754e5334b
querier for RestoreForState not closed. (#4922)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-28 15:25:17 +02:00
Krasi Georgiev 24520727a4
return an error when the last wal segment record is torn. (#451)
* return an error when the  last wal segment record is torn.

this ensures that a repair will be run when the last record in a segment
is torn.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-28 15:15:11 +02:00
Simon Pasquier fb32ef6000
Use Go modules (#454)
* *: support Go modules

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Update go.mod and Makefile.common

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-28 11:39:56 +01:00
Brian Brazil d50b9a5619
Reload after reading the WAL. (#460)
This causes the head to be GCed at startup,
removing any series that were read from the WAL
but have since been written to a block. In
systems with low ingestion rates, this potentially
could be many many hours of data.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-28 09:23:50 +00:00
Ben Kochie c6399296dc
Fix spelling/typos (#4921)
* Fix spelling/typos

Fix spelling/typos reported by codespell/misspell.
* UK -> US spelling changes.

Signed-off-by: Ben Kochie <superq@gmail.com>
2018-11-27 17:44:29 +01:00
Simon Pasquier 0bb810d126
discovery/marathon: fix leaked connections (#4915)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-27 14:58:27 +01:00
Stafford Williams 328d81999a /config: Added Copy to clipboard button for /config (#4410)
* added `Copy to clipboard` button

Signed-off-by: Stafford Williams <stafford.williams@gmail.com>

* generate vsfdata

Signed-off-by: Stafford Williams <stafford.williams@gmail.com>

* new lines

Signed-off-by: Stafford Williams <stafford.williams@gmail.com>

* single newline

Signed-off-by: Stafford Williams <stafford.williams@gmail.com>
2018-11-27 16:05:26 +08:00
Wei Guo e329cbf673 Add metric prometheus_rule_group_last_evaluation for recording and alerting (#4852)
* add metric prometheus_rule_group_last_evaluation for recording and alerting

Signed-off-by: Wei Guo <me@imkira.com>

* fix issues from comments

Signed-off-by: Wei Guo <me@imkira.com>
2018-11-27 14:38:13 +08:00
Brian Brazil d2f0f54d68
Pass through content-type for non-compressed output. (#4912)
Fixes #4911

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-26 13:05:07 +00:00
Timo Beckers bea302e061 marathon-sd - use 'hostPort' member of portMapping to construct target endpoints (#4887)
Fixes #4855 - ServicePort was wrongly used to construct an address to endpoints
defined in portMappings. This was changed to HostPort. Support for obtaining
auto-generated host ports was also added.

Signed-off-by: Timo Beckers <timo@incline.eu>
2018-11-26 13:39:35 +01:00
Frederic Branczyk f9c93b3686
Merge pull request #4905 from simonpasquier/cherrypick-4840
Cherrypick 4840
2018-11-26 10:35:02 +01:00
David Coles 1591ebf985 ui: Fix hover when metric has null value. (#4906)
When a metric has a null value, number formatters like
`humanizeNoSmallPrefix` will throw "Uncaught TypeError: Cannot read
property 'toPrecision' of null".

This is fixed by explicitly checking for `null` and returning the string
"null".

Note: This is usually not seen as rickshaw doesn't show annotations for
null values, but still calls the formatter.

Signed-off-by: David Coles <coles.david@gmail.com>
2018-11-25 14:16:48 +08:00
Fabian Reinartz a9803e9ecb Correctly skip mismatching targets
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-11-23 17:10:31 +01:00
Krasi Georgiev 080e6ed31a
collect cpu and trace profiles with the promtool debug command (#4897)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-23 17:57:31 +02:00
Daniele Sluijters f25a6baedb remote: Set User-Agent header in requests (#4891)
Currently Prometheus requests show up with a UA of Go-http-client/1.1
which isn't super helpful. Though the X-Prometheus-Remote-* headers
exist they need to be explicitly configured when logging the request in
order to be able to deduce this is a request originating from
Prometheus. By setting the header we remove this ambiguity and make
default server logs just a bit more useful.

This also updates a few other places to consistently capitalize the 'P'
in the user agent, as well as ensure we set a UA to begin with.

Signed-off-by: Daniele Sluijters <daenney@users.noreply.github.com>
2018-11-23 22:49:49 +08:00
Alex Yu 5dcce32ef8 update promlog to latest version (#4876)
* update promlog to latest version

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* Update api tests, fix main setup

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* tidy go.sum

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* revendor prometheus/common

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* only initialize config; use kingpin for remote_storage_adapter

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* actually parse the flags

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* clean up imports

Signed-off-by: Alex Yu <yu.alex96@gmail.com>
2018-11-23 14:22:40 +01:00
Wei Guo 996fd958ac fix deadlock in scrape manager (#4894)
Scrape manager will fall in deadlock when we reload configs frequently.
2018-11-23 11:23:55 +02:00
Brian Brazil 407e12d051 Make MemPostings nested.
This saves memory, about a quarter of the size of the postings map
itself with high-cardinality labels (not including the post ids).

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-22 09:36:39 +00:00
Brian Brazil fc99b8bb3a Make index reader postings nested.
This reduces memory by only having to store the string's 16
bytes+map overheard once per label name, rather than duplicating it in every
entry for the label value.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-22 09:36:39 +00:00
Brian Brazil c93e261466 Reduce memory taken up by posting/symbol tables.
Reuse the string already allocated for symbols
in the posting tables.

Use a slice for symbols in v2 format.

Move symbol size logic into the index code.
Avoid duplication of lookupSymbol logic.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-22 09:36:39 +00:00
Ganesh Vernekar cfb3769274 Lazily load samples for unit testing (#4851)
* Lazily load samples for unit testing

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* cleanup

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-11-22 14:21:38 +05:30
Tom Wilkie 88ebd749dd Make newBReader return a struct, not a pointer. (#459)
This shows up as a hot spot in profiles of queries involving lots of seeks, as each seek creates a new iterator.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-22 13:21:57 +05:30
arugaki b98a5acb57 fix buf redeclared in scrapeloop (#4873)
* buf Has been declared in scrape.go in line 785, I think it is unnecessary to declare a new variable again here.

Signed-off-by: arugakiWei <arugaki.wei@daocloud.io>

* delete the buf in line 785 because  it is never used.

Signed-off-by: arugakiWei <arugaki.wei@daocloud.io>
2018-11-22 12:13:52 +08:00
Krasi Georgiev bd100182b2
added tsdb/head mint maxt metrics (#4888)
added the head metrics with the correct suffix.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-21 12:57:32 +02:00
achiuBAE a9050c45f6 Allow setting the Prometheus instance document title through a flag. (#4841)
* web: added ability to set page title through flag.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* Reformatted variable names and Flag description for readability.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* assets_vfsdata.go

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* Flag name changed from web.ui-title to web.page-title

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* make assets

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>
2018-11-21 12:45:06 +08:00
Krasi Georgiev b75d702ceb
fix flaky compaction test (#458)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-21 00:33:14 +02:00
Simon Pasquier 87abb279e4
web: log stack traces on panic (#4221)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-20 14:25:03 +01:00
Krasi Georgiev 7f00217d77
Allow manual compaction for tests when compaction is disabled globally. (#412)
for tests we need to control when a compaction happens so with this
change automated compaction can be disabled, but allow to run it
manually it tests.

fixes failing tests in : https://github.com/prometheus/tsdb/pull/374

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-20 12:34:26 +02:00
Julius Volz a8b8a8d5df
Fix .PHONE -> .PHONY typo in Makefile.common (#4886)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-11-20 17:17:48 +08:00
Tom Wilkie 638204c775 Typo
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-19 12:23:42 +00:00
Sylvain Rabot 1fd3b33dcd Prevent Azure SD panic (fix #4779) (#4867)
Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>
2018-11-19 12:23:12 +00:00
Simon Pasquier ed19373a78
*: remove use of golang.org/x/net/context (#4869)
* *: remove use of golang.org/x/net/context

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* scrape: fix TestTargetScrapeScrapeCancel

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-19 12:31:16 +01:00
Tom Wilkie 8f42192e52 Add Prometheus alerts from kube-prometheus, remove the alertmanager alerts.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-19 11:22:55 +00:00
Ganesh Vernekar ca93fd544b /api/v1/labels endpoint for getting all label names (#4835)
* vendor: update tsdb

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* /api/v1/labels endpoint

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* regex matchers for API

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add docs

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Matchers behaving as OR

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Removed the matchers

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* vendor: update tsdb using go mod

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* vendor update: tsdb

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added LabelNames() to storage.Querier

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Test for api.labelNames

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-11-19 15:51:14 +05:30
Ganesh Vernekar 7f30395115 LabelNames() for Querier (#455)
* LabelNames() for Querier

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-11-16 19:02:24 +01:00
Tom Wilkie dfbdf8d3bb Add a basic readme with link to the mixin docs.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:23:14 +00:00
Tom Wilkie 5fd712b210 copypasta.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Tom Wilkie 50861d586a Alert if more than 1% of alerts fail for a given integration.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00