Commit graph

7491 commits

Author SHA1 Message Date
David Kaltschmidt 87c46ea6c3 Renamed TotalEvalTime to EvalTotalTime
* TotalFoo suggested a comprehensive timing, but TotalEvalTime was part
of the Exec timings, together with Queue timings
* The other option was to rename ExecTotalTime to TotalExecTime, but
 there was already ExecQueueTime, suggesting Exec to be some sort of
group
2017-11-17 17:46:51 +01:00
David Kaltschmidt af75ce02c1 Review feedback
* renamed MakeQueryStats
* added stats to query() as well
* gofmt
2017-11-16 16:30:48 +01:00
David Kaltschmidt c93e54d240 Adds execution timer stats to the range query
API consumers should be able to get insight into the query run times.
The UI currently measures total roundtrip times. This PR allows for more
fine grained metrics to be exposed.

* adds new timer for total execution time (queue + eval)

* expose new timer, queue timer, and eval timer in stats field of the
 range query response:
```json
{
  "status": "success",
  "data": {
    "resultType": "matrix",
    "result": [],
    "stats": {
      "execQueueTimeNs": 4683,
      "execTotalTimeNs": 2086587,
      "totalEvalTimeNs": 2077851
    }
  }
}
```

* stats field is optional, only set when query parameter `stats` is not
empty

Try it via
```sh
curl 'http://localhost:9090/api/v1/query_range?query=up&start=1486480279&end=1486483879&step=14000&stats=true'
```

Review feedback

* moved query stats json generation to query_stats.go
* use seconds for all query timers
* expose all timers available
* Changed ExecTotalTime string representation from Exec queue total time to Exec total time
2017-11-16 16:05:10 +01:00
Fabian Reinartz a031cf7424 Return ULID in Compactor 2017-11-14 15:25:30 +01:00
Goutham Veeramachaneni a00d700d48
Merge pull request #200 from jaxi/build_on_smartos
Ensure tsdb can be built on SmartOS/Illumos/Solaris
2017-11-14 19:28:25 +05:30
Tobias Schmidt a51c500e30
Merge pull request #3453 from prometheus/grobie/remote-read-filter
Add remote read filter option
2017-11-14 13:56:26 +01:00
Tobias Schmidt deb64bf79a
Merge pull request #3471 from Gouthamve/remove-skipped-evals
rules: remove skipped iterations because no throttling anymore
2017-11-14 13:08:35 +01:00
Goutham Veeramachaneni e1117715fe
rules: remove skipped iterations cuz no throttling
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-11-14 17:33:00 +05:30
Tobias Schmidt 7098c56474 Add remote read filter option
For special remote read endpoints which have only data for specific
queries, it is desired to limit the number of queries sent to the
configured remote read endpoint to reduce latency and performance
overhead.
2017-11-13 23:30:01 +01:00
Tobias Schmidt 434f0374f7 Refactor remote storage querier handling
* Decouple remote client from ReadRecent feature.
* Separate remote read filter into a small, testable function.
* Use storage.Queryable interface to compose independent
  functionalities.
2017-11-13 23:19:15 +01:00
Tobias Schmidt 9b0091d487 Add storage.Queryable and storage.QueryableFunc
In order to compose different querier implementations more easily, this
change introduces a separate storage.Queryable interface grouping the
query (Querier) function of the storage.

Furthermore, it adds a QueryableFunc type to ease writing very simple
queryable implementations.
2017-11-13 20:19:37 +01:00
Tobias Schmidt 91be55ebf0
Merge pull request #3458 from grandbora/test-race
Fix race in test
2017-11-13 17:57:21 +01:00
Bora Tunca 493fd6bd1f Fix race in test 2017-11-13 11:47:59 -05:00
Tobias Schmidt 6671277b12
Merge pull request #3464 from krasi-georgiev/flaky-file-discovery-tests-2
Fix flaky file discovery tests - sync the channel draining goroutine
2017-11-13 17:00:37 +01:00
Fabian Reinartz f1512a368a Expose ChunkSeriesSet and lookups methods. 2017-11-13 14:02:32 +01:00
Fabian Reinartz 3ef4326114 Refactor tombstone reader types 2017-11-13 13:38:07 +01:00
Krasi Georgiev 1005ef0a70 Fix flaky file discovery tests - sync the channel draining goroutine 2017-11-13 12:12:01 +00:00
Fabian Reinartz e5ce2bef43 Add explicit error to Querier.Select
This has been a frequent source of debugging pain since errors are
potentially delayed to a much later point. They bubble up in an
unrelated execution path.
2017-11-13 12:16:58 +01:00
Tobias Schmidt 5766b36be2
Merge pull request #3271 from dlespiau/remove-go-vendor-experiment
Remove mentions of GO15VENDOREXPERIMENT
2017-11-13 12:03:18 +01:00
Damien Lespiau 4fff98fd7f Remove last mention of GO15VENDOREXPERIMENT
Since #2750[1] prometheus depends on go 1.8. go 1.7 enabled vendoring by
default and removes the corresponding variable[2], simplifying things a bit.

[1] https://github.com/prometheus/prometheus/pull/2750
[2] https://blog.golang.org/go1.7
2017-11-13 11:00:30 +00:00
Ben Kochie 8bfbb4e733 Simplify Makefile promu fetch
Closes: https://github.com/prometheus/prometheus/issues/3460
2017-11-13 11:13:14 +01:00
Goutham Veeramachaneni 032413caf0
Merge pull request #205 from opsnull/master
fix typing
2017-11-13 10:52:10 +05:30
zhangjun3 65e6b79a7c fix typing 2017-11-13 13:10:23 +08:00
Bora Tunca 3cc01a3088 Add more discovery tests for updating target groups (#3426)
* Adds a test covering the case where a target providers sends updated versions of the same target groups and the system should reconcile to the latest version of each of the target groups
* Refactors how input data is represented in the tests. It used to be literal declarations of necessary structs, now it is parsing yaml. Yaml declarations are half as long as the former. And these can be put in a fixture file
* Adds a tiny bit of refactoring on test timeouts
2017-11-12 03:39:08 +01:00
Julius Volz 9f10c63cff
Fix remote read labelset corruption (#3456)
The labelsets returned from remote read are mutated in higher levels
(like seriesFilter.Labels()) and since the concreteSeriesSet didn't
return a copy, the external mutation affected the labelset in the
concreteSeries itself. This resulted in bizarre bugs where local and
remote series would show with identical label sets in the UI, but not be
deduplicated, since internally, a series might come to look like:

{__name__="node_load5", instance="192.168.1.202:12090", job="node_exporter", node="odroid", node="odroid"}

(note the repetition of the last label)
2017-11-12 00:47:47 +01:00
Krasi Georgiev c8a735ceb6 Fix flaky file discovery tests (#3438)
* flaky test caused by invalid fsnotify updates before the test files are written to disk causing the fd service to send empty `group[]` struct

* `close(filesReady)` needs to be before the file closing so that fsnotify triggers a new loop of the discovery service.

* nits

* use filepath.Join for the path url to be cross platform

* stupid mistake revert
2017-11-11 17:20:39 +01:00
Tobias Schmidt 83bb1becab
Merge pull request #3251 from krasi-georgiev/implement-go-kit-run-groups
Refactor main.go with  oklog/pkg/group actors pattern
2017-11-11 17:19:06 +01:00
Julius Volz 9703325901
Merge pull request #204 from prometheus/fix-wal-closing
Close WAL when closing the DB
2017-11-11 16:02:58 +01:00
Julius Volz 1dad3370fd Close WAL when closing the DB
Also, the `wal` field of the `DB` was not used anywhere, so this removes
it.
2017-11-11 14:56:23 +01:00
Krasi Georgiev e2f4850fea Refactor main.go with oklog/pkg/group actors pattern 2017-11-11 12:33:15 +00:00
Jorge Hernández 6cd0f63eb1 Use testutil in rules subpackage (#3278)
* Use testutil in rules subpackage

* Fix manager test

* Use testutil in rules subpackage

* Fix manager test

* Fix rebase

* Change to testutil for applyConfig tests
2017-11-11 11:29:47 +01:00
Alexey Miroshkin 8c681f4a6c Provide POST endpoint for query+query_range (#3322)
This PR fixes #3072 by providing POST endpoints for `query` and `query_range`.
POST request must be made with `Content-Type: application/x-www-form-urlencoded` header.
2017-11-11 01:53:48 +01:00
Fabian Reinartz 1d08a00d63
Merge pull request #203 from prometheus/absbytes
Generalize index and chunk readers
2017-11-10 14:10:41 +00:00
Fabian Reinartz ac5bd71d8f Doc fixes 2017-11-10 14:10:20 +00:00
Fabian Reinartz d578b10d5c chunk: make reader accept abstract ByteSlice 2017-11-10 10:38:22 +00:00
Fabian Reinartz b7c3cfecbf index: abstract ByteSlice and adjust indexReader
This replaces the builtin byte slice with an interface for the index
reader. This allows the complex decoding of the index file format
to be used against more generalized implementations.
2017-11-09 17:38:32 +00:00
Fabian Reinartz c354d6bd59 index: simplify checksum validation 2017-11-09 15:58:36 +00:00
Fabian Reinartz 798f2bdb0a
Merge pull request #189 from sunhay/checksum-checks
Index checksum validation on reads
2017-11-09 15:35:09 +00:00
Fabian Reinartz b4d093a8a7
Merge pull request #201 from prometheus/grobie/fix-typo-in-metric
Fix typo in prometheus_tsdb_reloads_failures_total description
2017-11-09 13:27:12 +00:00
Tobias Schmidt 5170198276 Fix typo in prometheus_tsdb_reloads_failures_total description 2017-11-09 14:25:27 +01:00
Jingkai He bfadf72d3f
Ensure tsdb can be built on SmartOS/Illumos/Solaris
At the moment tsdb cannot be built on Illumos/Solaris due to

- An exclude tag against Solaris is added in db_unix.go
- We use built-in syscall package, which doesn't have nmmap and munmap support

This PR supports build on Illumos/Solaris by remove the Solaris exclude tag in db_unix.go
At the same time use golang.org/x/sys/unix package instead of syscall package, since it has Solaris nmmap and munmap implementation.
2017-11-08 23:32:35 +00:00
Fabian Reinartz 1bacdf8e33
Merge pull request #198 from dswarbrick/master
Use Go 1.9 math/bits funcs for counting leading / trailing zeros
2017-11-08 23:31:49 +00:00
Daniel Swarbrick 57cefba0d5 Move to Go 1.9 2017-11-08 23:26:51 +01:00
Daniel Swarbrick 0585ab3fcd Use Go 1.9 math/bits implementation for counting leading / trailing zeros 2017-11-08 23:18:33 +01:00
Julius Volz 8022fc1aa3
Fix display of raw target labels (again) (#3437)
I already fixed this once in 7be94d6643,
but it seems it got broken again when merging other fixes from 1.x into
2.0.
2017-11-08 15:30:19 +01:00
Fabian Reinartz 7337c6a513
Merge pull request #196 from prometheus/grobie/fix-metric-prefix
Fix missing prefix of tsdb_wal_* metrics
2017-11-08 13:59:39 +00:00
Tobias Schmidt 6027af95ca Fix missing prefix of tsdb_wal_* metrics 2017-11-08 14:58:03 +01:00
Tobias Schmidt 6806b5fa2f Merge branch 'release-2.0' 2017-11-08 08:16:44 +01:00
Tom Wilkie 617e7d0203 Add migration docs for 2.0 (#3374)
* Initial draft of migration.md

* Edits.

* Review feedback.

* Review feedback.

* Staleness link to video; add docker root example; remote config file section.

* s/NB/NOTE/, remove external labels link.

* More typos.

* Add more details link for removed PromQL features.

* s/you/your/

* Expand on prom1.8/2.0 side by side setup.

* More feedback.

* update links.

* --query.lookback-delta flag.
2017-11-08 08:14:33 +01:00
Brian Brazil 0a74f98628 Release 2.0.0 2017-11-08 08:06:15 +01:00