Commit graph

927 commits

Author SHA1 Message Date
Julius Volz 63899f5529
Remove obsolete console.log() statements (#6159)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-10-17 16:24:07 +02:00
Julius Volz bca6e90ea6
Integrate beginning of React UI (#5694)
* Initial commit from Create React App

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Initial Prometheus expression browser code

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Grpahing, try out echarts

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Switch to flot

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Add metrics fetching and stuff

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Autosuggest and graph improvements

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Start implementing graph controls, add loading spinner

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* So many new features and fixes

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fixed and built more features

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Make datetimepicker clear work

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Don't abort when executing empty expression

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Remove TabPaneAlert

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Split components into separate files

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Add table time input

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Move first files to TypeScript!

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* More TypeScript conversions

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* More TS conversions

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* More TS conversions

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* More TS conversions

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* More TS conversions

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* More TS fixes

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Convert Graph to TS

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Changes

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Resize detector, start building legend, axis font colors

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Make graph legend work

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Add URL params support and much more

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Put panel state into panel list, write URL options

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Change order of Graph and Table tabs

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Generalize time input naming more

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Work on history functionality

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* npm updates

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Move loading indicator into "Execute" button

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fix typo

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Revert "Move loading indicator into "Execute" button"

This reverts commit ce7daee1f1af35da6c0d8b5517272839285ccfec.

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Improve error message when failing to fetch server time

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Move all code to Prometheus repo target dir

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Add react-app Makefile step and check in generated assets

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Add preliminary npm packages notice to NOTICE file

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Update React app's favicon and metadata

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Remove RP server refs, cleanups

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Use CircleCI image that includes NodeJS

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Add some missing React output assets

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Preserve CRLF in generated React files

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Switch from npm to yarn for React UI

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Save npm licenses and include them in release tarball

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Install npm on Travis

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Remove npm license tarball from source

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Remove React graph bundle from source

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Don't check in any compiled web assets

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Update README.md with node/yarn/React UI info

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fix asset build step on CircleCI promu crossbuild

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Try to fix multi-arch go generate

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Remove check_assets from Travis CI build

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Prevent rebuilding of unchanged React app parts

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fix npm license tarball path for promu

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Simplify Makefile

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Clarify build instructions in README.md

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Make minimal JS test pass

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Integrate React app tests into Makefile

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Separate react-app-tests target, but run it from CI

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fix working directory for React app tests

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Remove local modifications to Makefile.common

This means that CircleCI will not run the React app tests, but at least
Travis still will...

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Depend on node_modules path for npm_licenses target

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Simplify tarball/docker/build Makefile targets

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Include React tests in "test" target

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Remove reference to removed "check_assets" target

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Do initial resize of expression input field

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Add React app proxying to local Prometheus in dev mode

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-10-17 14:38:09 +02:00
Krasi Georgiev 81d284f806
Merge the 2.13 release branch to master (#6117) 2019-10-09 17:41:46 +02:00
Giedrius Statkevičius ad58fc4765 ui: handle null response properly (#6071)
* web/ui: handle null case

The call might sometimes return /api/v1/label/__name__/values the
following:

```
{"status":"success","data":null}
```

Then the `index.js` file assumes that `data` is not `null`. However,
that assumption fails and then we get this error in the console:

```
graph.js?v=foo:317 Uncaught TypeError: Cannot read property 'length' of null
    at Object.success (graph.js?v=foo:317)
...
```

Then it becomes impossible to, for example, send a simple query like
`time()` and graph the results.

Fix it by using an empty array as the result if it is `null`.

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

* ui: update static assets data

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
2019-10-08 19:52:12 -06:00
Sven Nebel 27bbc235e0 Improve message when Lifecycle API is not enabled
Signed-off-by: Sven Nebel <nebel.sven@gmail.com>
2019-09-25 10:57:21 +01:00
Sven Nebel d186d85649 Return 403 with PUT verb if lifecycle is not enabled
Signed-off-by: Sven Nebel <nebel.sven@gmail.com>
2019-09-25 10:57:11 +01:00
Yao Zengzeng 8744afdd1e cleanup redundant code of TestEndpoints (#6022)
Signed-off-by: YaoZengzeng <yaozengzeng@huawei.com>
2019-09-18 11:40:50 +01:00
Goutham Veeramachaneni 4c648eddf4
vendor: Update json-iterator (#6030)
Also add explicit support for NaN/Inf

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2019-09-17 19:22:26 +05:30
Simon Pasquier 26e8d25e0b
web/ui: avoid truncated Y-axis values (#6014)
* web/ui: avoid truncated Y-axis values

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix missing semi-colon

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-09-12 09:42:19 +02:00
Max cb5213e914 Change the global variable 'name' to a local variable so that it can not populate the global space. (#5970)
* Change the global variable 'name' to a  local variable so that it can not populate the global space.

Signed-off-by: 朱正浩,Zhu Zhenghao <zhenghao.zhu@daocloud.io>

* run make assets

Signed-off-by: 朱正浩,Zhu Zhenghao <zhenghao.zhu@daocloud.io>
2019-08-31 08:33:31 -06:00
Yao Zengzeng 21c9789083 multiple queries test for StreamRead (#5969)
Signed-off-by: YaoZengzeng <yaozengzeng@zju.edu.cn>
2019-08-29 11:57:38 +01:00
Stanislav Putrya 6141a8bd7c Show warnings in UI if query have returned some warnings (#5964)
* Show warnings in UI if query have returned some warnings
+ improve warning (error) text if query to remote was finished with error
* Add prefixes for remote_read errors

Signed-off-by: Stan Putrya <root.vagner@gmail.com>
2019-08-28 14:25:28 +01:00
Harkishen Singh f0bb8129c3 Fixes service-discovery view when accessed from unhealthy targets (#5915)
* fix service-discovery view

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>

* updated localStorage logic

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>

* sign off

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>

* committed vfs unicodes

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>
2019-08-21 13:57:08 +02:00
Bartek Płotka 48b2c9c8ea
remote-read: streamed chunked server side; Extended protobuf; Added chunked, checksumed reader (#5703)
Part of: https://github.com/prometheus/prometheus/issues/4517 and https://github.com/improbable-eng/thanos/issues/488

Changes:
* Extended protobuf for chunked remote read and negotation.
* Added checksumed, chunked Writer/Reader.
* Added Server side implementation for chunked streamed remote-read.


Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2019-08-19 21:16:10 +01:00
Ganesh Vernekar 5ecef3542d
Cleanup after merging tsdb into prometheus
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-08-13 14:04:14 +05:30
Chris Marchbanks 0685eb5395
Refactor testutil.NewStorage into a new package
This avoids a circular dependency between the testutil and storage
packages.

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2019-08-08 19:43:04 -06:00
slava 5e974d3170 Fix context for the showWarning function (#5833)
* Fix context for the showWarning function

If the difference between the current time on a client and time on a server is quite big, Prometheus tries to show a related warning in UI on the Graph tab. But in the code, an incorrect context is used to invoke this method. As a result, an error is showed in the web developer console and the whole page stop working at all. This commit fixes the context.

CC @juliusv

Signed-off-by: Vyacheslav Kulakov <vkulakov@swiftserve.com>

* Fix context for the showWarning function

Fixed assets

CC @juliusv

Signed-off-by: Vyacheslav Kulakov <vkulakov@swiftserve.com>
2019-08-05 20:03:12 +02:00
@aifsair 0f00737308 Fix log config setup (#5807)
The return value of Set() was not checked.
Hence, the typo ("al" instead of "af") wasn't catched.

Signed-off-by: François (fser) <fser@code-libre.org>
2019-07-29 18:00:30 +01:00
Tariq Ibrahim 9fe9e66cfd
remove the dependency on cockroachdb in prometheus
Signed-off-by: Tariq Ibrahim <tariq181290@gmail.com>
2019-07-09 01:27:17 -07:00
Thomas Jackson fef150f1b5 Add tests to ensure we can marshal and unmarshal our min/max times (#5734)
* Add tests to ensure we can marshal and unmarshal our min/max times

Related to https://github.com/prometheus/client_golang/issues/614

Instead of implementing all the time parsing, we can special-case handle
these 2 times. This means if times in this format show up that
time.Parse can't handle they will still error, but we can marshal/parse
our own min/max time

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>
2019-07-08 10:43:59 +01:00
Sylvain Rabot 7cd5cf0b69 Sanitize HTML output (#5665)
* Compact html output
* Fix html style block location

Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>
2019-06-25 13:39:34 +01:00
Simon Pasquier be67b8d460
web: fix flaky TestHTTPMetrics() (#5695)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-06-24 15:48:15 +02:00
Simon Pasquier d7f38dfdde
web: add 'code' label to HTTP metrics (#5640)
* web: add prometheus_http_requests_total metrics

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Add unit test for requestCounter metric

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-06-19 17:56:04 +02:00
Thomas Jackson 91d7175eaa Add storage.Warnings to LabelValues and LabelNames (#5673)
Fixes #5661

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>
2019-06-17 08:31:17 +01:00
Alex Salt d6a4daa26a web api: handle alert with Infinity/NaN values (#5582)
* web/api/v1: alert value as string in alert/rules endpoints

Signed-off-by: Alexander Saltykov <alexander-s@yandex-team.ru>
2019-05-21 10:41:54 +01:00
Tariq Ibrahim c4b57ac4e2
fix minor issue in the <a> tag of rules.html
Signed-off-by: Tariq Ibrahim <tariq181290@gmail.com>
2019-05-16 13:00:35 -07:00
pbhudiaBAE 43953b105b Sorting alerts by group name in /alerts (#5448)
* Working group name

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Working categorised by group name

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Changed group sorting in web

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Fixed group sorting and comments

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Fixed group sorting and comments with gofmt

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Added file and group name

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* reverted back to full path to yml file

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>
2019-05-14 23:14:27 +02:00
mrasu 3a7a743bc4 Consistent Content-Type for js and css (#5551)
Currently, When `/etc/mime.types` has a unusual mime type, web of prometheus uses the type and you may get unexpected result.
With this change, web returns consistent Content-Type header for static js and css files

To reproduce:
1. Add a type at the end of `/etc/mime` like `text/x-js js`
2. Run prometheus
3. Request js file like `http://localhost:9090/static/vendor/js/jquery.min.js`
4. You will see Content-Type of the response is `text/x-js` instead of `application/javascript`

Signed-off-by: mrasu <m.rasu.hitsuji@gmail.com>
2019-05-09 16:10:15 +01:00
Simon Pasquier 45506841e6
*: enable all default linters (#5504)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-05-03 15:11:28 +02:00
Simon Pasquier 03b90b4645
web/ui: update to Bootstrap 4.3.1 (#5506)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-30 10:16:36 +02:00
Bjoern Rabenstein 38d518c0fe Rework #5009 after comments
Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-17 01:40:10 +02:00
Bjoern Rabenstein a92ef68dd8 Fix staticcheck errors
Not sure why they only show up now.

Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-17 01:40:10 +02:00
Sylvain Rabot 335a34486e Add external labels to template expansion
This affects the expansion of templates in alert labels and
annotations and console templates.

Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>
2019-04-17 01:40:10 +02:00
Simon Pasquier 81c4248081
*: bump gRPC and protobuf dependencies (#5367)
The goal is to remove almost all references to the
golang.org/x/net/context package.

github.com/gogo/protobuf => v1.2.1
google.golang.org/grpc => v1.19.1
github.com/grpc-ecosystem/grpc-gateway => v1.18.5

It also replaces github.com/cockroachdb/cmux by github.com/soheilhy/cmux
because of [1] which fixes #3909 incidentally.

[1] https://github.com/grpc/grpc-go/issues/2636

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-04 11:55:32 +02:00
Bob Shannon 8c8bb82d04 Add support for POSTing to /series endpoint (#5422)
* Add support for POSTing to /series endpoint
* Document query API POST support

Signed-off-by: Bob Shannon <bob.m.shannon@gmail.com>
2019-04-02 18:00:29 +01:00
Ganesh Vernekar 0afd172224
Merge pull request #5416 from prometheus/release-2.8
Merge 2.8.1 into master
2019-03-29 15:20:49 +05:30
Simon Pasquier 7328cb6cc2 web/ui: display the job label in /targets
PR #4806 was removed accidentally by #5192 (Bootstrap upgrade).

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-03-26 16:52:48 +01:00
Tariq Ibrahim 8fdfa8abea refine error handling in prometheus (#5388)
i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors.
ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives.
iii) Does away with the use of fmt package for errors in favour of pkg/errors

Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-26 00:01:12 +01:00
Bharath 91306bdf24 Support non POST methods for Lifecycle and Admin APIs (#5376)
Signed-off-by: Bharath Thiruveedula <bharath_ves@hotmail.com>
2019-03-20 17:33:45 +00:00
Matthias Loibl b7e978d255 web/ui: Fix padding in navbar container (#5371)
Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>
2019-03-18 23:17:49 +01:00
Tom Wilkie c7b3535997 Use pkg/relabelling in remote write.
- Unmarshall external_labels config as labels.Labels, add tests.
- Convert some more uses of model.LabelSet to labels.Labels.
- Remove old relabel pkg (fixes #3647).
- Validate external label names.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-03-18 20:31:12 +00:00
Julien Pivotto 6c72cdb1e1 Add GODEBUG to the runtime page (#5324)
This is useful, e.g. with go 1.12 to see if TLS1.3 has been enabled.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-03-18 11:10:13 +00:00
Tariq Ibrahim d6f0e14ac6 fix redundant err!=nil check in web.go (#5346)
Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-12 16:25:46 +00:00
Krasi Georgiev 9d96ada510 Display correct values for the retention in the flags web gui. (#5322)
* Display correct values for the retention in the flags web gui.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* adding a log entry

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* added the retention info to the runtime status page

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* simplify the retention display

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-03-11 22:48:57 +05:30
Julien Pivotto 6152df44c2 ui: Expand expression_select to 220px
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-03-11 17:24:25 +01:00
Ganesh Vernekar 46e0587cb3
Merge pull request #5318 from roidelapluie/sdtag
ui: remove extra table tag in service discovery
2019-03-11 20:13:13 +05:30
Julien Pivotto 5a162dc1ab
ui: remove extra table tag in service discovery
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-03-09 11:02:14 +01:00
Julien Pivotto 981f9208fb
ui: Remove time picker borders
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-03-08 22:58:31 +01:00
David 38fea83c1f Debounce input key press handling (#5309)
- input key handler causes 2 layout cycles on each keypress which can
clog up browser rendering when typing quickly
- this change adds a debounce to the key press handler of 500ms

Fixes #5308
Signed-off-by: David Kaltschmidt <david.kaltschmidt@gmail.com>
2019-03-06 16:16:55 +01:00
Julius Volz 795c989d36 Merge branch 'master' into bootstrap4
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-19 22:32:55 +00:00
Palash Nigam 09208b1a58 queryRange: Add more descriptive error messages (#5229)
Fixes: https://github.com/prometheus/prometheus/issues/4811

Signed-off-by: Palash Nigam <npalash25@gmail.com>
2019-02-19 19:16:14 +00:00
Julius Volz fdbaef86df Re-add typeahead license header to minified file
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-17 22:50:20 +00:00
Julius Volz 661f0127bc Rebuild web assets
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-17 22:50:16 +00:00
Julius Volz ed635190ba Re-add popper.js to fix target label tooltips
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-17 19:40:57 +00:00
Julius Volz 7b724cea3a Whitespace and other cleanups
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-17 19:40:52 +00:00
Julius Volz 7244ef3783 Add more top/bottom spacing for All/Unhealthy buttons
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-17 19:40:48 +00:00
Julius Volz 028e99e3d6 Remove spacing between All/Unhealthy buttons
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-17 19:40:44 +00:00
Julius Volz 45b91e8e80 Fix copy&paste button on /config, move pre style to CSS
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-02-17 19:40:30 +00:00
Julius Volz cd569b51d9 Merge branch 'master' into bootstrap4 2019-02-17 17:22:41 +00:00
Callum Styan 6f69e31398 Tail the TSDB WAL for remote_write
This change switches the remote_write API to use the TSDB WAL.  This should reduce memory usage and prevent sample loss when the remote end point is down.

We use the new LiveReader from TSDB to tail WAL segments.  Logic for finding the tracking segment is included in this PR.  The WAL is tailed once for each remote_write endpoint specified. Reading from the segment is based on a ticker rather than relying on fsnotify write events, which were found to be complicated and unreliable in early prototypes.

Enqueuing a sample for sending via remote_write can now block, to provide back pressure.  Queues are still required to acheive parallelism and batching.  We have updated the queue config based on new defaults for queue capacity and pending samples values - much smaller values are now possible.  The remote_write resharding code has been updated to prevent deadlocks, and extra tests have been added for these cases.

As part of this change, we attempt to guarantee that samples are not lost; however this initial version doesn't guarantee this across Prometheus restarts or non-retryable errors from the remote end (eg 400s).

This changes also includes the following optimisations:
- only marshal the proto request once, not once per retry
- maintain a single copy of the labels for given series to reduce GC pressure

Other minor tweaks:
- only reshard if we've also successfully sent recently
- add pending samples, latest sent timestamp, WAL events processed metrics

Co-authored-by: Chris Marchbanks <csmarchbanks.com> (initial prototype)
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com> (sharding changes)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-02-12 11:39:13 +00:00
Maria Nemtinova 8e3a39f725 Web UI QoL improvements (#5201)
1. Added an ability to resize text area on mouseclick
2. Remember selected target status button on page reload

Signed-off-by: Maria Nemtinova <nemtinovamasha@gmail.com>
2019-02-12 00:22:05 +01:00
Minh-Long Do b26b5c9e96 Add rendering test of template based web endpoints (#5188)
Signed-off-by: Minh-Long  Do <minhlong.langos@gmail.com>
2019-02-08 10:17:47 +00:00
jritchieBAE b8f0a41745 Update to Bootstrap 4.1.3 (#5192)
* web: updated bootstrap3-typeahead file to work with bootstrap 4.0.0

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: Replaced bootstrap-3.3.1 with bootstrap 4.0.0

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: Added bootstrap4-glyphicons as 4.0.0 doesnt include bootstrap3 glyphicons

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: updated js jquery to 3.3.1

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: updated _base.html to import new bootstrap 4.0.0, jquery3.3.1 and bootstrap class tags to be 4.0 compatible

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: _base.html missed word out in title tag (Server).

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: updated alerts.html class names and tags to be bootstrap 4 compatible.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: updated config.html class names and tags to be bootstrap 4 compatible.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: updated flags.html class names and tags to be bootstrap 4 compatible.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: updated service-discovery.html class names and tags to be bootstrap 4 compatible.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: updated status.html class names and tags to be bootstrap 4 compatible.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: updated targets.html class names and tags to be bootstrap 4 compatible.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: updated graph_template.handlebar class names and tags to be bootstrap 4 compatible.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: alerts.css fix for button color inheritance on alerts page.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: graph.css fix for color inheritance.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: prometheus.css updated to fix nav bar.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* web: previous merge conflict not fixed correctly on _base.html

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* menu.lib and prom.lib imports updated

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* bootstrap 4.1.3 imported

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Bootstrap 4.1.3 imported into _base.html

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* bootstrap 4.1.3 imported into prom.lib

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* menu.lib style adjusted to view sidebar

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Alert colour uplifted to bootstrap 4.1.3

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Alerts display code reformatted similarly to config

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Consoles pages adjusted to account for new navbar

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* LHS Menu fixed in console pages

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Minor changes to prom_console to adjust lhs nav

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Prom.lib and some css updated to fix console graph controls

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Bootstrap 4.0.0 files removed

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Consoles configured so that the graph fits with the new side bar, css files also adjusted

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Import popper.min.js for dropdowns

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Popper.min.js imported locally

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Re-added #4764 and fixed css

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Removed .DS_Store

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Rebuilt assets

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* Spaces between buttons and inputs on graph page removed

Signed-off-by: ksherryBAE <kieran.sherry@baesystems.com>

* fixed spacing in buttons on /targets

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Updated vfsdata.go

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* fixed typeahead issue

Signed-off-by: James Ritchie <james.g.ritchie@baesystems.com>

* added css for dropdown

Signed-off-by: James Ritchie <james.g.ritchie@baesystems.com>

* changed order of css imports

Signed-off-by: James Ritchie <james.g.ritchie@baesystems.com>

* tinkered with CSS changes to make keyboard select and mouseover match

Signed-off-by: James Ritchie <james.g.ritchie@baesystems.com>
2019-02-07 22:18:09 +01:00
zhulongcheng fd964426a7 web: predeclare and reuse errors (#5180)
Predeclare and reuse errors to reduce duplicate code

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-02-04 13:06:26 +01:00
zhulongcheng a75f8a8e05 update error message in extractTimeRange (#5179)
Update error message in the extractTimeRange function
to match function's logic

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-02-03 09:29:23 +00:00
Simon Pasquier a60431f3cd Merge v2.7.1 into master (#5170)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-01 09:54:12 +01:00
Vishnunarayan K I 108b9b0e5f Limit number of merics in prometheus UI (#5139)
Signed-off-by: Vishnunarayan K I <appukuttancr@gmail.com>
2019-01-31 17:03:50 +00:00
Hrishikesh Barman a1f34bec2e Added CORS Origin flag (#5011)
Signed-off-by: Hrishikesh Barman <hrishikeshbman@gmail.com>
2019-01-17 15:01:06 +00:00
Matt Layher 302148fd69 *: apply gofmt -s
Signed-off-by: Matt Layher <mdlayher@gmail.com>
2019-01-16 17:28:14 -05:00
Callum Styan 5358f76c5c update remote write path proto so that Labels/Timeseries can't be nil (#4957)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-01-15 19:13:39 +00:00
Simon Pasquier 375ad1185c
*: bump gRPC dependencies (#5075)
* *: bump gRPC dependencies

This change updates the gRPC dependencies to more recent versions:

* github.com/gogo/protobuf => v1.2.0
* github.com/grpc-ecosystem/grpc-gateway => v1.6.3
* google.golang.org/grpc => v1.17.0

In addition scripts/genproto.sh leverages Go modules information instead of
hardcoding SHA1 commits. This ensures that the code is generated from
the exact same sources.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Run 'make proto' in CI

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Revert tabs -> spaces change

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix 'make proto' step

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* 'go get' grpc/protobuf dependencies

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Prepopulate cache with go mod download

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-15 15:32:05 +01:00
Simon Pasquier f678e27eb6
*: use latest release of staticcheck (#5057)
* *: use latest release of staticcheck

It also fixes a couple of things in the code flagged by the additional
checks.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Use official release of staticcheck

Also run 'go list' before staticcheck to avoid failures when downloading packages.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-04 14:47:38 +01:00
Tom Wilkie 6e08029b56
Move err to be the last return value from storage.Select. (#5054)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-01-02 11:10:13 +00:00
SenXuDC 8fd0c0ab2e fix typo imeplements -> implements (#4979)
Signed-off-by: SenXuDC <sen.xu@daocloud.io>
2018-12-18 11:52:16 +01:00
Julius Volz 11a52be1d8
Better rounding for incoming query timestamps (#4941)
Fixes https://github.com/prometheus/prometheus/issues/4939

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-12-03 20:25:54 +08:00
mknapphrt f0e9196dca Return warnings on a remote read fail (#4832)
Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>
2018-11-30 14:27:12 +00:00
Simon Pasquier 77ee41084f
web: group targets by job then instance (#4898)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-30 15:24:15 +01:00
Simon Pasquier 632626b553
web: deduplicate handler label for HTTP metrics (#4752)
* web: deduplicate handler label for HTTP metrics

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* remove instrumentHandlerFunc

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-30 13:30:32 +01:00
Ben Kochie c6399296dc
Fix spelling/typos (#4921)
* Fix spelling/typos

Fix spelling/typos reported by codespell/misspell.
* UK -> US spelling changes.

Signed-off-by: Ben Kochie <superq@gmail.com>
2018-11-27 17:44:29 +01:00
Simon Pasquier 0bb810d126
discovery/marathon: fix leaked connections (#4915)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-27 14:58:27 +01:00
Stafford Williams 328d81999a /config: Added Copy to clipboard button for /config (#4410)
* added `Copy to clipboard` button

Signed-off-by: Stafford Williams <stafford.williams@gmail.com>

* generate vsfdata

Signed-off-by: Stafford Williams <stafford.williams@gmail.com>

* new lines

Signed-off-by: Stafford Williams <stafford.williams@gmail.com>

* single newline

Signed-off-by: Stafford Williams <stafford.williams@gmail.com>
2018-11-27 16:05:26 +08:00
Frederic Branczyk f9c93b3686
Merge pull request #4905 from simonpasquier/cherrypick-4840
Cherrypick 4840
2018-11-26 10:35:02 +01:00
David Coles 1591ebf985 ui: Fix hover when metric has null value. (#4906)
When a metric has a null value, number formatters like
`humanizeNoSmallPrefix` will throw "Uncaught TypeError: Cannot read
property 'toPrecision' of null".

This is fixed by explicitly checking for `null` and returning the string
"null".

Note: This is usually not seen as rickshaw doesn't show annotations for
null values, but still calls the formatter.

Signed-off-by: David Coles <coles.david@gmail.com>
2018-11-25 14:16:48 +08:00
Fabian Reinartz a9803e9ecb Correctly skip mismatching targets
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-11-23 17:10:31 +01:00
Alex Yu 5dcce32ef8 update promlog to latest version (#4876)
* update promlog to latest version

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* Update api tests, fix main setup

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* tidy go.sum

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* revendor prometheus/common

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* only initialize config; use kingpin for remote_storage_adapter

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* actually parse the flags

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* clean up imports

Signed-off-by: Alex Yu <yu.alex96@gmail.com>
2018-11-23 14:22:40 +01:00
achiuBAE a9050c45f6 Allow setting the Prometheus instance document title through a flag. (#4841)
* web: added ability to set page title through flag.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* Reformatted variable names and Flag description for readability.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* assets_vfsdata.go

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* Flag name changed from web.ui-title to web.page-title

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* make assets

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>
2018-11-21 12:45:06 +08:00
Simon Pasquier 87abb279e4
web: log stack traces on panic (#4221)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-20 14:25:03 +01:00
Ganesh Vernekar ca93fd544b /api/v1/labels endpoint for getting all label names (#4835)
* vendor: update tsdb

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* /api/v1/labels endpoint

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* regex matchers for API

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add docs

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Matchers behaving as OR

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Removed the matchers

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* vendor: update tsdb using go mod

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* vendor update: tsdb

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added LabelNames() to storage.Querier

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Test for api.labelNames

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-11-19 15:51:14 +05:30
Simon Pasquier 6fa8de132b
web/v1/api: add tests for admin actions (#4767)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-15 14:22:16 +01:00
Frederic Branczyk 5464c64853
Merge pull request #4802 from simonpasquier/go-modules
Switch to Go modules
2018-11-15 11:43:44 +01:00
Goutham Veeramachaneni 7acedbce64
web(api): Make query and range api errors match
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2018-11-14 15:25:54 +05:30
Simon Pasquier 181f07ef26
web: avoid proxy to connect to the local gRPC server (#4572)
By default the gRPC client of the REST API gateway relies on the
HTTP_PROXY variable to connect to the local gRPC server which isn't
desired as the server runs in the same process. This change uses a
custom dialer that connects directly to the server's address.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-13 14:42:23 +01:00
Simon Pasquier d89c01b065 web/ui: fix asset generation for Go modules installs
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-12 10:56:21 +01:00
Simon Pasquier bf9a1d3d15 *: fix assets generation
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-12 10:38:29 +01:00
Simon Pasquier 0a319b001a Update documentation
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-09 15:01:15 +01:00
Simon Pasquier 5230e2730a
*: check that assets are up-to-date (#4731)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-09 11:31:36 +01:00
Simon Pasquier 147f2061d5 web: display job label in targets page
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-10-30 16:20:43 +01:00
Frederic Branczyk ad5139fcba
Merge pull request #4783 from simonpasquier/fix-targets-api
web/api/v1: fix targets endpoint
2018-10-26 15:37:12 +02:00
Wei Guo 742583e836 Support to query elements by a specific time (#4764)
* support to query elements by a specific time

Signed-off-by: Wei Guo <me@imkira.com>
2018-10-26 14:16:40 +02:00
Simon Pasquier a308a186e4 web/api/v1: fix targets endpoint
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-10-25 11:10:26 +02:00
Brian Brazil 9c03e11c2c Hook OpenMetrics parser into scraping.
Extend metadata api to support units.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-10-18 13:58:00 +01:00
Ariya Hidayat a0ba9b7f3e Avoid browser's spell-checking the Expression field (#4728)
Signed-off-by: Ariya Hidayat <ariya.hidayat@gmail.com>
2018-10-17 10:54:16 +02:00
Simon Pasquier c4a6acfb1e
*: move to go 1.11 (#4626)
* *: move to go 1.11

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Reduce number of places where we specify the Go version

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-10-16 09:41:45 +02:00
Goutham Veeramachaneni ffb7f829ec
Merge pull request #4730 from prometheus/release-2.4
Release 2.4
2018-10-12 14:15:42 -07:00
Will Hegedus 193ebe7e34 Updates to /targets and /rules (scrape duration, last evaluation time) (#4722)
* Add evaluationTimestamp (Last Evaluation) column to display on /rules
Signed-off-by: Will Hegedus <wbhegedus@liberty.edu>

* Add lastScrapeDuration ("Scrape Duration") to display on /targets
Signed-off-by: Will Hegedus <wbhegedus@liberty.edu>

* Updates based on Julius' feedback

Signed-off-by: Will Hegedus <wbhegedus@liberty.edu>

* Update to set timestamp to when eval started (after eval completes)

Signed-off-by: Will Hegedus <wbhegedus@liberty.edu>

* Update /rules to display time since last evaluation

Signed-off-by: Will Hegedus <wbhegedus@liberty.edu>

* Re-order Last Eval/Eval Time to be consistent with targets page

Signed-off-by: Will Hegedus <wbhegedus@liberty.edu>
2018-10-12 18:26:59 +02:00
Tariq Ibrahim d371697841 Adding new metric type to track in-flight queries via the remote read API endpoint. (#4699)
* Adding new metric type to track in-flight queries via the remote read API endpoint.

Signed-off-by: tariqibrahim <tariq.ibrahim@microsoft.com>

* fix review comments

Signed-off-by: tariqibrahim <tariq.ibrahim@microsoft.com>

* fix comments

Signed-off-by: tariqibrahim <tariq.ibrahim@microsoft.com>
2018-10-10 16:09:08 -07:00
Benji Visser eb4cc37e50 Update moment.js and moment-timezone.js (#4679)
* upgrading moment-timezome (0.5.13 -> 0.5.17)
* upgrading dependency moment (2.16.0 -> 2.22.2)

Signed-off-by: noqcks <benny@noqcks.io>
2018-10-10 09:48:06 +01:00
Julien Pivotto 17fea5478a Improve rule views by wrapping lines (#4702)
With the addition of the errors in the views list, it is now difficult
to have a view on all the rules in a screen witdh.

This commit adds wrapping to improve the overall display of the rules
page.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2018-10-07 15:31:16 +01:00
Callum Styan 9bca041285 WIP: keep track of samples per query, set a max # of samples (#4513)
* keep track of samples per query, set a max # of samples that can be in
memory at once

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2018-10-02 12:59:19 +01:00
Simon Pasquier 3c00eeaf16 web/api/v1: fix optional skip_head for snapshot (#4674)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-10-02 16:18:07 +05:30
Krasi Georgiev 47a673c3a0
process scrape loops reloading in parallel (#4526)
The scrape manage receiver's channel now just saves the target sets
and another backgorund runner updates the scrape loops every 5 seconds.
This is so that the scrape manager doesn't block the receiving channel
when it does the long background reloading of the scrape loops.

Active and dropped targets are now saved in each scrape pool instead of
the scrape manager. This is mainly to avoid races when getting the
targets via the web api.

When reloading the scrape loops now happens in parallel to speed up the
final disared state and this also speeds up the prometheus's shutting
down.

Also updated some funcs signatures in the web package for consistency.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-09-26 12:20:56 +03:00
Tom Wilkie 4c52400708
Limit concurrent remote reads. (#4656)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-09-25 20:07:34 +01:00
Tom Wilkie d3a1ff1abf
Reduce memory usage of remote read by reducing pointer usage. (#4655)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-09-25 19:14:00 +01:00
Goutham Veeramachaneni 1a5a3d3be6
Merge pull request #4645 from prometheus/release-2.4
Merge release-2.4 into master
2018-09-21 15:38:12 +05:30
Simon Pasquier 2d7f562ed6 web: fix asset paths for Windows platforms (#4616)
* web: fix asset paths for Windows platforms

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* web: add tests

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-09-19 12:50:53 +05:30
Goutham Veeramachaneni 9ee4b05d16
Better link for docs (#4593)
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2018-09-13 15:26:41 +05:30
Goutham Veeramachaneni 3e87c04b83
Logger is nil for API. Fixes #4577 (#4583)
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2018-09-06 16:07:48 +05:30
Tom Wilkie 457e4bb58e
Limit the number of samples remote read can return. (#4532)
* Limit the number of samples remote read can return.

- Return 413 entity too large.
- Limit can be set be a flag.  Allow 0 to mean no limit.
- Include limit in error message.
- Set default limit to 50M (* 16 bytes = 800MB).

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-09-05 15:50:50 +02:00
Simon Pasquier 75bd348135 web: clean up api/v2 (#4554)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-08-29 12:55:46 +05:30
Thomas Jackson f0a1ebc19d Add SelectParams to Select for federation (#4546)
When prom2 came out the storage querier interface consolidated to a
single Select() method. While doing this it makes it impossible as the
implementer of the querier to know if you are being called for metadata
or actual data. The workaround has been to check if the SelectParams are
nil, which the federation call is always nil. This has 2 negative
consequences (1) remote implementations interpret this as a metadata
call, which makes the federation endpoint return nothing. (2) this means
that the storage implementations don't get the same information passed
down to them as far as SelectParams goes.

This diff simply adds SelectParams to the Select() call in the
federation handler

Mitigation for #4057

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>
2018-08-28 11:23:31 +01:00
Simon Pasquier 3581377e5d Replace go-bindata with vfsgen (#4430)
Looking at https://tech.townsourced.com/post/embedding-static-files-in-go/ (which was mentioned in the issue), vfsgen has all the needed features.

In particular:

- Reproducible builds (no issue with timestamping).
- Well maintained and relatively popular.
- Integration with go generate.
- Self-contained (no external dependency).

* [WIP] Replace go-bindata by vfsgen

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Add license + remove doc.go

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Generate templates assets

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Use new templates assets

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* split static assets

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Idempotent make assets

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Update vendor/

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* vendor vfsgendev

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Update README.md

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Simplify assets generation

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix README.md

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Use generate helper program instead of vfsgen

This avoids installing vfsgendev in the target environment.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Remove unused vfsgen package

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix Makefile

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* vendoring shurcooL/vfsgen

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix go generate command

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Sync web/ui/assets_vfsdata.go

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-08-24 09:03:10 +02:00
Max Inden ecf676cf97 web/api: Expose rule health and last error (#4501)
Expose rule health and last evaluation error on `/api/v1/rules`.

Signed-off-by: Max Leonard Inden <IndenML@gmail.com>
2018-08-23 18:30:10 +05:30
Julius Volz 8fbe1b5133
Handle a bunch of unchecked errors (#4461)
There are many more (mostly finalizers like Close/Stop/etc.), but most of
the others seemed like one couldn't do much about them anyway.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-08-17 17:24:35 +02:00
Benji Visser 8bb6e0dd6e Show rule evaluation errors on rules page (#4457)
* adding information about the health and errors for Rules

adding Health() and LastError() to the Rule interface. This will allow
us to easily surface information about rules.

Signed-off-by: noqcks <benny@noqcks.io>

* updating rules.html with fields for Rule errors and health state

Signed-off-by: noqcks <benny@noqcks.io>

* fix code comment grammar & access Rule health/error info using a mutex

Signed-off-by: noqcks <benny@noqcks.io>

* s/Errors/Error/ in rules.html to remain consistent with targets.html

Signed-off-by: noqcks <benny@noqcks.io>

* adding periods to code comments in reporting/alerting

Signed-off-by: noqcks <benny@noqcks.io>

* putting health/error below mutex in struct field

Signed-off-by: noqcks <benny@noqcks.io>
2018-08-07 00:33:45 +02:00
Julius Volz 159e1537d2
Remove /heap endpoint (#4460)
It was added 5 years ago by Matt and I'm not sure anyone ever used
it after public release (since we have /debug/pprof/heap as well).

It also lacked error checking and allows people to write to disk over HTTP.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-08-04 21:31:28 +02:00
Ganesh Vernekar f1db699dff Persist alert 'for' state across restarts (#4061)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-08-02 11:18:24 +01:00
Max Leonard Inden 71fafad099
api/v1: Coninue work exposing rules and alerts
Signed-off-by: Max Leonard Inden <IndenML@gmail.com>
2018-07-30 15:31:51 +02:00
mg03 31f8ca0dfb
api v1 alerts/rules json endpoint
Signed-off-by: mg03 <mgeng03@gmail.com>
2018-07-30 15:29:44 +02:00
Julius Volz 8763dedb9d
Update method name in rules template, fix rendering (#4416)
Fixes https://github.com/prometheus/prometheus/issues/4407

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-07-25 18:25:43 +02:00
Tom Wilkie b1f600343f
Merge pull request #4359 from prometheus/report-errors
Log errors encountered when marshalling and writing responses.
2018-07-25 13:39:04 +01:00
Tom Wilkie 02534510ca Review feedback.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-07-25 13:35:47 +01:00
Tom Wilkie 901e6d1f82 Review feedback.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-07-25 13:17:10 +01:00
Stafford Williams 286a8e0554 go-bindata debug clarification (#4411)
Signed-off-by: Stafford Williams <stafford.williams@gmail.com>
2018-07-24 14:01:56 +01:00
Adam Shannon 566c80b47c web: add named anchors for each rule group (#4130)
* web: add named anchors for each rule group

Signed-off-by: Adam Shannon <adamkshannon@gmail.com>
2018-07-18 16:34:41 +01:00
Alin Sinpalean e3b775b78b Simplify BufferedSeriesIterator usage (#4294)
* Allow for BufferedSeriesIterator instances to be created without an underlying iterator, to simplify their usage.

Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>
2018-07-18 05:10:28 +01:00
Martin Lee d0f11a3cc6 Forbid rule-abiding robots from indexing. (#4266)
* Resolves github issue #4257

Signed-off-by: Martin Lee <martin@billforward.net>
2018-07-18 05:01:57 +01:00
Thomas Jackson 92c6f0c92e Add offset to selectParams (#4226)
* Add Start/End to SelectParams
* Make remote read use the new selectParams for start/end

This commit will continue sending the start/end time of the remote read
query as the overarching promql time and the specific range of data that
the query is intersted in receiving a response to is now part of the
ReadHints (upstream discussion in #4226).

* Remove unused vendored code

The genproto.sh script was updated, but the code wasn't regenerated.
This simply removes the vendored deps that are no longer part of the
codegen output.

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>
2018-07-18 04:58:00 +01:00
Tom Wilkie f83155b11e Review feedback.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-07-13 19:31:23 +01:00
Tom Wilkie ccb2ee607b Log errors encountered when marshalling and writing responses.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-07-06 18:44:45 +01:00
Tom Wilkie fcc3f43acd spelling.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-06-18 17:32:44 +01:00
Tom Wilkie ae29512444 Extend API tests to cover remote read API.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-06-18 17:21:12 +01:00
Fabian Reinartz e7cfc7dae5 web: restore old path prefix behavior
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-15 09:08:14 +01:00
Fabian Reinartz 7a74689973
Merge pull request #4258 from prometheus/rmsecheaders
web: remove security headers
2018-06-12 16:21:35 +02:00
Fabian Reinartz 8a4bda8d57 web: remove security headers
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-12 08:59:13 -04:00
Fabian Reinartz 057a5ae2b1 Address comments
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-06 11:21:17 -04:00
Fabian Reinartz ad4c33c1ff scrape,api: provide per-target metric metadata
This adds a per-target cache of scraped metadata. The metadata is only
available for the lifecycle of the attached target. An API endpoint allows
to select metadata by metric name and a label selection of targets.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-06 05:56:10 -04:00
Fabian Reinartz 9c83e815d1 Merge branch 'release-2.2' into update 2018-06-05 05:29:11 -04:00
Simon Pasquier 097439b0f9 web: limit the number of dropped targets (#4212)
Displaying all the dropped targets in the service-discovery page hurts
the Prometheus server as well as the browser when thousands of dropped
targets exist. This change limits this number to 1,000 and display the
number of active/total targets per scrape configuration.

Add warning when more than 100 targets are dropped

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-05 10:20:32 +01:00
Brian Brazil dd6781add2 Optimise PromQL (#3966)
* Move range logic to 'eval'

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make aggregegate range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* PromQL is statically typed, so don't eval to find the type.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Extend rangewrapper to multiple exprs

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Start making function evaluation ranged

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make instant queries a special case of range queries

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Eliminate evalString

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Evaluate range vector functions one series at a time

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make unary operators range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make binops range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Pass time to range-aware functions.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make simple _over_time functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reduce allocs when working with matrix selectors

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add basic benchmark for range evaluation

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse objects for function arguments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Do dropmetricname and allocating output vector only once.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add range-aware support for range vector functions with params

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise holt_winters, cut cpu and allocs by ~25%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make rate&friends range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make more functions range aware. Document calling convention.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make date functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make simple math functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Convert more functions to be range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make more functions range aware

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Specialcase timestamp() with vector selector arg for range awareness

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove transition code for functions

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove the rest of the engine transition code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove more obselete code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove the last uses of the eval* functions

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove engine finalizers to prevent corruption

The finalizers set by matrixSelector were being called
just before the value they were retruning to the pool
was then being provided to the caller. Thus a concurrent query
could corrupt the data that the user has just been returned.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add new benchmark suite for range functinos

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Migrate existing benchmarks to new system

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Expand promql benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Simply test by removing unused range code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* When testing instant queries, check range queries too.

To protect against subsequent steps in a range query being
affected by the previous steps, add a test that evaluates
an instant query that we know works again as a range query
with the tiimestamp we care about not being the first step.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse ring for matrix iters. Put query results back in pool.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse buffer when iterating over matrix selectors

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Unary minus should remove metric name

Cut down benchmarks for faster runs.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reduce repetition in benchmark test cases

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Work series by series when doing normal vectorSelectors

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise benchmark setup, cuts time by 60%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Have rangeWrapper use an evalNodeHelper to cache across steps

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Use evalNodeHelper with functions

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Cache dropMetricName within a node evaluation.

This saves both the calculations and allocs done by dropMetricName
across steps.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse input vectors in rangewrapper

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Reuse the point slices in the matrixes input/output by rangeWrapper

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make benchmark setup faster using AddFast

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Simplify benchmark code.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add caching in VectorBinop

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Use xor to have one-level resultMetric hash key

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Add more benchmarks

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Call Query.Close in apiv1

This allows point slices allocated for the response data
to be reused by later queries, saving allocations.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise histogram_quantile

It's now 5-10% faster with 97% less garbage generated for 1k steps

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make the input collection in rangeVector linear rather than quadratic

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise label_replace, for 1k steps 15x fewer allocs and 3x faster

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Optimise label_join, 1.8x faster and 11x less memory for 1k steps

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Expand benchmarks, cleanup comments, simplify numSteps logic.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Address Fabian's comments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Comments from Alin.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Address jrv's comments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Remove dead code

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Address Simon's comments.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Rename populateIterators, pre-init some sizes

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Handle case where function has non-matrix args first

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Split rangeWrapper out to rangeEval function, improve comments

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Cleanup and make things more consistent

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Make EvalNodeHelper public

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>

* Fabian's comments.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-06-04 15:47:45 +02:00
Henri DF 2952387ed1 Pass query hints down into remote read query proto (#4122)
Signed-off-by: Henri DF <henridf@gmail.com>
2018-05-08 09:48:13 +01:00