Commit graph

4553 commits

Author SHA1 Message Date
David Kaltschmidt 0a3faee526 Pushed query timings into timings field, added tests 2017-11-17 22:30:59 +01:00
David Kaltschmidt 87c46ea6c3 Renamed TotalEvalTime to EvalTotalTime
* TotalFoo suggested a comprehensive timing, but TotalEvalTime was part
of the Exec timings, together with Queue timings
* The other option was to rename ExecTotalTime to TotalExecTime, but
 there was already ExecQueueTime, suggesting Exec to be some sort of
group
2017-11-17 17:46:51 +01:00
David Kaltschmidt af75ce02c1 Review feedback
* renamed MakeQueryStats
* added stats to query() as well
* gofmt
2017-11-16 16:30:48 +01:00
David Kaltschmidt c93e54d240 Adds execution timer stats to the range query
API consumers should be able to get insight into the query run times.
The UI currently measures total roundtrip times. This PR allows for more
fine grained metrics to be exposed.

* adds new timer for total execution time (queue + eval)

* expose new timer, queue timer, and eval timer in stats field of the
 range query response:
```json
{
  "status": "success",
  "data": {
    "resultType": "matrix",
    "result": [],
    "stats": {
      "execQueueTimeNs": 4683,
      "execTotalTimeNs": 2086587,
      "totalEvalTimeNs": 2077851
    }
  }
}
```

* stats field is optional, only set when query parameter `stats` is not
empty

Try it via
```sh
curl 'http://localhost:9090/api/v1/query_range?query=up&start=1486480279&end=1486483879&step=14000&stats=true'
```

Review feedback

* moved query stats json generation to query_stats.go
* use seconds for all query timers
* expose all timers available
* Changed ExecTotalTime string representation from Exec queue total time to Exec total time
2017-11-16 16:05:10 +01:00
Tobias Schmidt a51c500e30
Merge pull request #3453 from prometheus/grobie/remote-read-filter
Add remote read filter option
2017-11-14 13:56:26 +01:00
Tobias Schmidt deb64bf79a
Merge pull request #3471 from Gouthamve/remove-skipped-evals
rules: remove skipped iterations because no throttling anymore
2017-11-14 13:08:35 +01:00
Goutham Veeramachaneni e1117715fe
rules: remove skipped iterations cuz no throttling
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-11-14 17:33:00 +05:30
Tobias Schmidt 7098c56474 Add remote read filter option
For special remote read endpoints which have only data for specific
queries, it is desired to limit the number of queries sent to the
configured remote read endpoint to reduce latency and performance
overhead.
2017-11-13 23:30:01 +01:00
Tobias Schmidt 434f0374f7 Refactor remote storage querier handling
* Decouple remote client from ReadRecent feature.
* Separate remote read filter into a small, testable function.
* Use storage.Queryable interface to compose independent
  functionalities.
2017-11-13 23:19:15 +01:00
Tobias Schmidt 9b0091d487 Add storage.Queryable and storage.QueryableFunc
In order to compose different querier implementations more easily, this
change introduces a separate storage.Queryable interface grouping the
query (Querier) function of the storage.

Furthermore, it adds a QueryableFunc type to ease writing very simple
queryable implementations.
2017-11-13 20:19:37 +01:00
Tobias Schmidt 91be55ebf0
Merge pull request #3458 from grandbora/test-race
Fix race in test
2017-11-13 17:57:21 +01:00
Bora Tunca 493fd6bd1f Fix race in test 2017-11-13 11:47:59 -05:00
Tobias Schmidt 6671277b12
Merge pull request #3464 from krasi-georgiev/flaky-file-discovery-tests-2
Fix flaky file discovery tests - sync the channel draining goroutine
2017-11-13 17:00:37 +01:00
Krasi Georgiev 1005ef0a70 Fix flaky file discovery tests - sync the channel draining goroutine 2017-11-13 12:12:01 +00:00
Tobias Schmidt 5766b36be2
Merge pull request #3271 from dlespiau/remove-go-vendor-experiment
Remove mentions of GO15VENDOREXPERIMENT
2017-11-13 12:03:18 +01:00
Damien Lespiau 4fff98fd7f Remove last mention of GO15VENDOREXPERIMENT
Since #2750[1] prometheus depends on go 1.8. go 1.7 enabled vendoring by
default and removes the corresponding variable[2], simplifying things a bit.

[1] https://github.com/prometheus/prometheus/pull/2750
[2] https://blog.golang.org/go1.7
2017-11-13 11:00:30 +00:00
Bora Tunca 3cc01a3088 Add more discovery tests for updating target groups (#3426)
* Adds a test covering the case where a target providers sends updated versions of the same target groups and the system should reconcile to the latest version of each of the target groups
* Refactors how input data is represented in the tests. It used to be literal declarations of necessary structs, now it is parsing yaml. Yaml declarations are half as long as the former. And these can be put in a fixture file
* Adds a tiny bit of refactoring on test timeouts
2017-11-12 03:39:08 +01:00
Julius Volz 9f10c63cff
Fix remote read labelset corruption (#3456)
The labelsets returned from remote read are mutated in higher levels
(like seriesFilter.Labels()) and since the concreteSeriesSet didn't
return a copy, the external mutation affected the labelset in the
concreteSeries itself. This resulted in bizarre bugs where local and
remote series would show with identical label sets in the UI, but not be
deduplicated, since internally, a series might come to look like:

{__name__="node_load5", instance="192.168.1.202:12090", job="node_exporter", node="odroid", node="odroid"}

(note the repetition of the last label)
2017-11-12 00:47:47 +01:00
Krasi Georgiev c8a735ceb6 Fix flaky file discovery tests (#3438)
* flaky test caused by invalid fsnotify updates before the test files are written to disk causing the fd service to send empty `group[]` struct

* `close(filesReady)` needs to be before the file closing so that fsnotify triggers a new loop of the discovery service.

* nits

* use filepath.Join for the path url to be cross platform

* stupid mistake revert
2017-11-11 17:20:39 +01:00
Tobias Schmidt 83bb1becab
Merge pull request #3251 from krasi-georgiev/implement-go-kit-run-groups
Refactor main.go with  oklog/pkg/group actors pattern
2017-11-11 17:19:06 +01:00
Krasi Georgiev e2f4850fea Refactor main.go with oklog/pkg/group actors pattern 2017-11-11 12:33:15 +00:00
Jorge Hernández 6cd0f63eb1 Use testutil in rules subpackage (#3278)
* Use testutil in rules subpackage

* Fix manager test

* Use testutil in rules subpackage

* Fix manager test

* Fix rebase

* Change to testutil for applyConfig tests
2017-11-11 11:29:47 +01:00
Alexey Miroshkin 8c681f4a6c Provide POST endpoint for query+query_range (#3322)
This PR fixes #3072 by providing POST endpoints for `query` and `query_range`.
POST request must be made with `Content-Type: application/x-www-form-urlencoded` header.
2017-11-11 01:53:48 +01:00
Julius Volz 8022fc1aa3
Fix display of raw target labels (again) (#3437)
I already fixed this once in 7be94d6643,
but it seems it got broken again when merging other fixes from 1.x into
2.0.
2017-11-08 15:30:19 +01:00
Tobias Schmidt 6806b5fa2f Merge branch 'release-2.0' 2017-11-08 08:16:44 +01:00
Tom Wilkie 617e7d0203 Add migration docs for 2.0 (#3374)
* Initial draft of migration.md

* Edits.

* Review feedback.

* Review feedback.

* Staleness link to video; add docker root example; remote config file section.

* s/NB/NOTE/, remove external labels link.

* More typos.

* Add more details link for removed PromQL features.

* s/you/your/

* Expand on prom1.8/2.0 side by side setup.

* More feedback.

* update links.

* --query.lookback-delta flag.
2017-11-08 08:14:33 +01:00
Brian Brazil 0a74f98628 Release 2.0.0 2017-11-08 08:06:15 +01:00
Julius Volz 02ca988bbd Remove /api/v1/delete_series docs for 2.0 (#3425)
This endpoint has moved to /api/v2 (with somewhat different properties)
in Prometheus 2.0 and should now be part of a separate admin API page.
2017-11-07 22:37:03 +00:00
Tobias Schmidt a117f051da Remove outdated information about next-release doc branch 2017-11-07 22:28:04 +01:00
Julius Volz ef08df0e6f
Add 2.0 storage docs (#3423)
* Add 2.0 storage docs

* Review fixups

* More review fixups
2017-11-07 22:00:38 +01:00
Matthew Pound e6dcc72f9a Fix instructions on updating prometheus.yml for Remote Write Adapter Example (#3422) 2017-11-07 20:02:43 +00:00
Krasi Georgiev 08a012ccd7 Fix for #3312 sigpanic in TargetManager (#3420) 2017-11-07 15:56:00 +00:00
Brian Brazil f689476b33
Update consoles for new metric names. (#3327) 2017-11-07 13:23:45 +00:00
Bora Tunca e63219ae6a Add discovery test (#3417) 2017-11-06 17:33:52 +00:00
Bora Tunca 09be10a553 Add test to prove redundant calls to identical target providers (#3404) 2017-11-06 16:14:15 +00:00
James Turnbull 4db6592d01 Removing external_labels from example conf file (#3409)
It's unclear why this is in the example configuration file. Probably best to keep that super simple, c.f. https://github.com/prometheus/docs/pull/895#discussion_r148924390
2017-11-06 16:11:04 +00:00
Sylvain Rabot db5b1aa5f6 Makefile: Allow $(GO) to be set with environmental variable (#2919)
* promu: Use same go invocation as prometheus' for building

If you override $(GO) var in prometheus Makefile in order to use
a custom go binary you might encounter errors while building promu
as promu's Makefile $(GO) var is also defined to look for go in the
$PATH and you might end up using different go binaries for building
prometheus and promu.

This patch makes prometheus Makefile pass its $(GO) var to promu's makefile
as environmental variable.

Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>

* Makefile: Allow $(GO) to be set with environmental variable

Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>

* Makefile: Fix hardcoded call to gofmt

Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>
2017-11-06 15:55:43 +01:00
David e3b926c03b Fix typo in AM config field static_configs (#3415)
* typo in prometheus.yml field causes prometheus to throw an error

Fixes #3414
2017-11-06 09:46:09 +00:00
Björn Rabenstein 8a9b32d0eb
Merge pull request #3410 from prometheus/beorn7/release
Cut 2.0.0-rc.3
2017-11-04 21:26:05 +01:00
beorn7 36a6ed4fe1 Cut v2.0.0-rc.3 2017-11-04 19:31:20 +01:00
beorn7 463faf5816 Ignore staticcheck's SA1019 in discovery/kubernetes/kubernetes.go, too
This showed up all of a sudden, even if checking out older commits.
2017-11-04 17:41:39 +01:00
beorn7 139929a363 Update vendoring of prometheus/tsdb and prometheus/common 2017-11-04 17:26:17 +01:00
beorn7 348ea482ea Merge branch 'beorn7/release' 2017-11-04 12:32:49 +01:00
beorn7 d7e9cca287 Cut 1.8.2 2017-11-04 11:49:35 +01:00
Björn Rabenstein bf8e002fce
Merge pull request #3405 from dominikschulz/fix/issue-3001
Guard against tags being nil in EC2 discovery
2017-11-03 17:20:40 +01:00
Dominik Schulz a731a43302 Guard against tags being nil in EC2 discovery
Fixes #3001
2017-11-03 13:23:01 +01:00
Björn Rabenstein 1829681e84
Merge pull request #3400 from prometheus/beorn7/release
Update vendoring for tsdb and clean it up in general
2017-11-03 13:09:19 +01:00
Björn Rabenstein c686f40adf
Merge pull request #3396 from prometheus/beorn7/doc
Remove @beorn7 from MAINTAINERS.md, promote @fabxc to storage maintainer
2017-11-03 12:56:16 +01:00
beorn7 d8c58d17e0 Remove _local storage_ from fabxc's responsibilities again
This is just for now, as fabxc doesn't seem to be available to agree
to his new role. Just revert this commit once he has.
2017-11-03 12:52:24 +01:00
beorn7 8093122782 Clean up vendoring
This doesn't perform any actual updates. (While that is something that
we should consider eventually, it leads down a rabbit hole and would
be too risky just before the 2.0.0 release.)

This is really just a clean-up. In particular:

- The following packages are not really used and thus removed:
  - github.com/blang/semver
  - github.com/coreos/go-oidc/http
  - github.com/coreos/go-oidc/jose
  - github.com/coreos/go-oidc/key
  - github.com/coreos/go-oidc/oauth2
  - github.com/coreos/go-oidc/oidc
  - github.com/coreos/pkg/health
  - github.com/coreos/pkg/httputil
  - github.com/coreos/pkg/timeutil
  - github.com/gophercloud/gophercloud/openstack/compute/v2/extensions
  - github.com/pborman/uuid

- For unclear reasons, the checksum for the following packages were
  wrong. The only change in this commit is to adjust the checksum in
  vendor.json. No changes of source code involved.
  - github.com/cespare/xxhash
  - github.com/dgryski/go-bits
  - github.com/golang/snappy

- Various packages from the github.com/gogo/protobuf repository are
  used throughout the code. They are vendored from various sources
  (different sub-vendoring of cockroachdb/cockroach and
  k8s.io/client-go) and revisions. I made them all consistently
  referring to the current version of the github.com/gogo/protobuf
  repository. This _is_ a change of code, but a necessary one to clean
  up the inconsistencies.

- For the github.com/influxdata/influxdb/... packages, the license
  files were missing. Also, the revision time was wrong, again for
  unknown reason.

This tries to avoid any real updates (as that would lead down a rabbit
hole, although I think we should check what dependencies need an
update one day) but at least tries to remove unused packages.
2017-11-02 17:50:00 +01:00