Commit graph

7444 commits

Author SHA1 Message Date
Krasi Georgiev 080e6ed31a
collect cpu and trace profiles with the promtool debug command (#4897)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-23 17:57:31 +02:00
Daniele Sluijters f25a6baedb remote: Set User-Agent header in requests (#4891)
Currently Prometheus requests show up with a UA of Go-http-client/1.1
which isn't super helpful. Though the X-Prometheus-Remote-* headers
exist they need to be explicitly configured when logging the request in
order to be able to deduce this is a request originating from
Prometheus. By setting the header we remove this ambiguity and make
default server logs just a bit more useful.

This also updates a few other places to consistently capitalize the 'P'
in the user agent, as well as ensure we set a UA to begin with.

Signed-off-by: Daniele Sluijters <daenney@users.noreply.github.com>
2018-11-23 22:49:49 +08:00
Alex Yu 5dcce32ef8 update promlog to latest version (#4876)
* update promlog to latest version

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* Update api tests, fix main setup

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* tidy go.sum

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* revendor prometheus/common

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* only initialize config; use kingpin for remote_storage_adapter

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* actually parse the flags

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* clean up imports

Signed-off-by: Alex Yu <yu.alex96@gmail.com>
2018-11-23 14:22:40 +01:00
Wei Guo 996fd958ac fix deadlock in scrape manager (#4894)
Scrape manager will fall in deadlock when we reload configs frequently.
2018-11-23 11:23:55 +02:00
Brian Brazil 407e12d051 Make MemPostings nested.
This saves memory, about a quarter of the size of the postings map
itself with high-cardinality labels (not including the post ids).

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-22 09:36:39 +00:00
Brian Brazil fc99b8bb3a Make index reader postings nested.
This reduces memory by only having to store the string's 16
bytes+map overheard once per label name, rather than duplicating it in every
entry for the label value.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-22 09:36:39 +00:00
Brian Brazil c93e261466 Reduce memory taken up by posting/symbol tables.
Reuse the string already allocated for symbols
in the posting tables.

Use a slice for symbols in v2 format.

Move symbol size logic into the index code.
Avoid duplication of lookupSymbol logic.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-22 09:36:39 +00:00
Ganesh Vernekar cfb3769274 Lazily load samples for unit testing (#4851)
* Lazily load samples for unit testing

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* cleanup

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-11-22 14:21:38 +05:30
Tom Wilkie 88ebd749dd Make newBReader return a struct, not a pointer. (#459)
This shows up as a hot spot in profiles of queries involving lots of seeks, as each seek creates a new iterator.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-22 13:21:57 +05:30
arugaki b98a5acb57 fix buf redeclared in scrapeloop (#4873)
* buf Has been declared in scrape.go in line 785, I think it is unnecessary to declare a new variable again here.

Signed-off-by: arugakiWei <arugaki.wei@daocloud.io>

* delete the buf in line 785 because  it is never used.

Signed-off-by: arugakiWei <arugaki.wei@daocloud.io>
2018-11-22 12:13:52 +08:00
Krasi Georgiev bd100182b2
added tsdb/head mint maxt metrics (#4888)
added the head metrics with the correct suffix.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-21 12:57:32 +02:00
achiuBAE a9050c45f6 Allow setting the Prometheus instance document title through a flag. (#4841)
* web: added ability to set page title through flag.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* Reformatted variable names and Flag description for readability.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* assets_vfsdata.go

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* Flag name changed from web.ui-title to web.page-title

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* make assets

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>
2018-11-21 12:45:06 +08:00
Krasi Georgiev b75d702ceb
fix flaky compaction test (#458)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-21 00:33:14 +02:00
Simon Pasquier 87abb279e4
web: log stack traces on panic (#4221)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-20 14:25:03 +01:00
Krasi Georgiev 7f00217d77
Allow manual compaction for tests when compaction is disabled globally. (#412)
for tests we need to control when a compaction happens so with this
change automated compaction can be disabled, but allow to run it
manually it tests.

fixes failing tests in : https://github.com/prometheus/tsdb/pull/374

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-20 12:34:26 +02:00
Julius Volz a8b8a8d5df
Fix .PHONE -> .PHONY typo in Makefile.common (#4886)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-11-20 17:17:48 +08:00
Tom Wilkie 638204c775 Typo
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-19 12:23:42 +00:00
Sylvain Rabot 1fd3b33dcd Prevent Azure SD panic (fix #4779) (#4867)
Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>
2018-11-19 12:23:12 +00:00
Simon Pasquier ed19373a78
*: remove use of golang.org/x/net/context (#4869)
* *: remove use of golang.org/x/net/context

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* scrape: fix TestTargetScrapeScrapeCancel

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-19 12:31:16 +01:00
Tom Wilkie 8f42192e52 Add Prometheus alerts from kube-prometheus, remove the alertmanager alerts.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-19 11:22:55 +00:00
Ganesh Vernekar ca93fd544b /api/v1/labels endpoint for getting all label names (#4835)
* vendor: update tsdb

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* /api/v1/labels endpoint

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* regex matchers for API

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add docs

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Matchers behaving as OR

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Removed the matchers

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* vendor: update tsdb using go mod

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* vendor update: tsdb

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added LabelNames() to storage.Querier

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Test for api.labelNames

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-11-19 15:51:14 +05:30
Ganesh Vernekar 7f30395115 LabelNames() for Querier (#455)
* LabelNames() for Querier

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-11-16 19:02:24 +01:00
Tom Wilkie dfbdf8d3bb Add a basic readme with link to the mixin docs.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:23:14 +00:00
Tom Wilkie 5fd712b210 copypasta.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Tom Wilkie 50861d586a Alert if more than 1% of alerts fail for a given integration.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Tom Wilkie 266ba185fe Remove PromScrapeFailed alert.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Tom Wilkie e8a8ce5654 Basic Prometheus dashboard.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Tom Wilkie ee1427faad Prometheus monitoring mixin for Prometheus itself.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Benji Visser 11b336e3ca Migrate all Docker image references to Docker Hub (#4864)
Signed-off-by: noqcks <benny@noqcks.io>
2018-11-16 11:26:10 +00:00
Junyu Wang 2f93fa6292 Included default flags into Dockerfile entrypoint (#4796)
* included default flags into Dockerfile entrypoint

Signed-off-by: Junyu Wang <junyuw@berkeley.edu>

* keeping style consistent

Co-Authored-By: DrakeW <junyuw@berkeley.edu>
Signed-off-by: Junyu Wang <junyuw@berkeley.edu>

* move prometheus.yml config into entrypoint & point to console template/libraries in etc/prometheus

Signed-off-by: Junyu Wang <junyuw@berkeley.edu>
2018-11-16 09:17:32 +01:00
Simon Pasquier 6fa8de132b
web/v1/api: add tests for admin actions (#4767)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-15 14:22:16 +01:00
Brian Brazil 41b54585d9
Use already open blocks while compacting. (#441)
This roughly halves the RAM requirements of compaction.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-15 12:20:54 +00:00
Frederic Branczyk 5464c64853
Merge pull request #4802 from simonpasquier/go-modules
Switch to Go modules
2018-11-15 11:43:44 +01:00
stuart nelson 6a69471bc2
[promtool] Support writing output as json (#4848)
* Support writing output as json

Oftentimes I'll want to execute something based on
the output from promtool, and supporting json
makes it easy to pull out values with a supporting
tool such as jq.

Signed-off-by: stuart nelson <stuartnelson3@gmail.com>
2018-11-14 18:40:07 +01:00
Krasi Georgiev 3385571ddf
buffer-panic when reading a record after recPageTerm (#429)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-14 18:43:33 +02:00
Krasi Georgiev 5a9ddeecef
fix lint errors (#439)
unexported NewMemTombstones as this returns unexported memTombstones
type which will not be shows in godoc.
Added missing comments for exported methods.
Removed unused RecordLogger,RecordReader interfaces.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-11-14 18:40:01 +02:00
Frederic Branczyk b82f930238
Merge pull request #4865 from squat/buffersignals
cmd/prometheus: buffer signal chans
2018-11-14 16:32:23 +01:00
Brian Brazil 910f3021b0
Use sampleBuf instead of maintaining lastValue. (#444)
This cuts the size of memSize by 8B.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-14 14:02:32 +00:00
Brian Brazil 10632217ce
Merge pull request #440 from prometheus/wal-reading
Improve WAL reading
2018-11-14 13:59:41 +00:00
nilsocket 80981a6aac FromMap(), sorts and returns instead of calling New() (#433)
Signed-off-by: nilsocket <nilsocket@gmail.com>
2018-11-14 13:43:03 +01:00
Goutham Veeramachaneni d4b348a7b1
Merge pull request #4866 from gouthamve/api-compat
web(api): Make query and range api errors match
2018-11-14 13:26:07 +01:00
Goutham Veeramachaneni 7acedbce64
web(api): Make query and range api errors match
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2018-11-14 15:25:54 +05:30
Goutham Veeramachaneni 1f053f897b
Merge pull request #4628 from bboreham/fix-storage-error
Make ErrStorage a concrete type not an interface
2018-11-14 10:39:16 +01:00
Lucas Serven 70c8b2c63c
cmd/prometheus: buffer signal chans
According to the GoDoc for os.Signal [0]:

> Package signal will not block sending to c: the caller must ensure that
> c has sufficient buffer space to keep up with the expected signal rate.
> For a channel used for notification of just one signal value, a buffer
> of size 1 is sufficient.

[0] https://golang.org/pkg/os/signal/#Notify

Signed-off-by: Lucas Serven <lserven@gmail.com>
2018-11-14 10:33:28 +01:00
Simon Pasquier 181f07ef26
web: avoid proxy to connect to the local gRPC server (#4572)
By default the gRPC client of the REST API gateway relies on the
HTTP_PROXY variable to connect to the local gRPC server which isn't
desired as the server runs in the same process. This change uses a
custom dialer that connects directly to the server's address.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-13 14:42:23 +01:00
fengyuceNv 94fff219ab improve remote storage enqueue performance (#4772)
Signed-off-by: fyc <fyc22788@ly.com>
2018-11-13 12:19:05 +00:00
Brian Brazil 8edaa8ad4d
Fix goroutine leak in lexer/parser. (#4858)
When there was an error in the parser, the
lexer goroutine was left running.

Also make runtime panic test actually test things.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-11-12 18:47:13 +00:00
Simon Pasquier d89c01b065 web/ui: fix asset generation for Go modules installs
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-12 10:56:21 +01:00
Simon Pasquier bf9a1d3d15 *: fix assets generation
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-12 10:38:29 +01:00
Simon Pasquier 5dce2f5853 Merge branch 'master' into go-modules
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-12 09:57:53 +01:00