Commit graph

7921 commits

Author SHA1 Message Date
Tobias Schmidt e6cdc2d355 Import querying documentation from prometheus/docs 2017-10-26 22:36:47 +02:00
Sunny Klair b65dd43c5b Validate index series/postings/symbol table checksums on read 2017-10-26 15:34:31 -04:00
Sunny Klair ab02ea4de4 Validate index offset table checksums on read 2017-10-26 13:48:31 -04:00
Tom Wilkie 323556b025 Merge pull request #3357 from prometheus/revert-3339-3065-remote-read-bypass
Revert "Bypass the fanout storage merging if no remote storage is configured."
2017-10-26 16:15:50 +01:00
Tom Wilkie 0e572686db Revert "Bypass the fanout storage merging if no remote storage is configured." 2017-10-26 16:09:39 +01:00
Tobias Schmidt 299802dfd0 Integrate changes from prometheus/docs 2017-10-26 16:14:43 +02:00
Tobias Schmidt 41281aff81 Include 1.8 changes in configuration docs 2017-10-26 16:14:43 +02:00
Tobias Schmidt 53a5f52224 Import first batch of Prometheus documentation
In order to provide documentation for each individual version, this
commit starts moving Prometheus server specific documentation into the
repository itself.
2017-10-26 16:14:43 +02:00
Tom Wilkie 7aa2930ac5 Merge pull request #3336 from tomwilkie/remote-api-update
Update remote storage APIs from 1.8 branch
2017-10-26 13:55:51 +01:00
Tom Wilkie 1af3ef431d s/TestRemoveLabels/TestSeriesSetFilter/ 2017-10-26 13:50:39 +01:00
Tom Wilkie 9c3c98e8de Revert "Port 'Don't disable HTTP keep-alives for remote storage connections.' to 2.0 (see #3173)"
This reverts commit 0997191b18.
2017-10-26 13:43:48 +01:00
Tom Wilkie 746752b946 Merge external labels in order. 2017-10-26 11:44:49 +01:00
Tom Wilkie 6e4d4ea402 Initialise some counters in remote storage API. 2017-10-26 11:09:45 +01:00
Tom Wilkie b22485bef0 Remove spurious test import. 2017-10-26 11:09:43 +01:00
Tom Wilkie 2ae04d0e79 Add license header. 2017-10-26 11:09:16 +01:00
Tom Wilkie e8c264e47a Add comment. 2017-10-26 11:09:16 +01:00
Tom Wilkie ee011d906d Port remote read server to 2.0. 2017-10-26 11:09:14 +01:00
Bryan Boreham 0997191b18 Port 'Don't disable HTTP keep-alives for remote storage connections.' to 2.0 (see #3173)
Removes configurability introduced in #3160 in favour of hard-coding,
per advice from @brian-brazil.
2017-10-26 11:08:33 +01:00
Tom Wilkie 56820726fa Move a couple of the encoding/decoding functions into codec.go 2017-10-26 11:08:33 +01:00
Conor Broderick 08b7328669 Port Metric name validation to 2.0 (see #2975) 2017-10-26 11:08:33 +01:00
Tom Wilkie 8fe0212ff7 Port 'Make queue manager configurable.' to 2.0, see #2991 2017-10-26 11:08:33 +01:00
Tom Wilkie 3760f56c0c remote: Expose ClientConfig type (see #3165) 2017-10-26 11:08:33 +01:00
Tom Wilkie 16f71a7723 Port codec.go over form 1.8 branch. 2017-10-26 11:08:33 +01:00
Fabian Reinartz e53040e2ac Merge pull request #3339 from tomwilkie/3065-remote-read-bypass
Bypass the fanout storage merging if no remote storage is configured.
2017-10-26 09:14:26 +02:00
Fabian Reinartz bf6db1bc3b Merge pull request #3344 from criteo-forks/master
Tsdb StartTime : Use a simplier way to compute StartTime
2017-10-26 09:08:48 +02:00
Fabian Reinartz bf56ad4233 Merge branch 'master' into master 2017-10-26 09:06:12 +02:00
Fabian Reinartz 3055b55830 Merge pull request #3354 from pgier/tsdb-block-duration-check
storage/tsdb: check that max block duration is larger than min
2017-10-26 09:04:54 +02:00
Fabian Reinartz 5800c257af Merge pull request #3355 from prometheus/remove-codeclimate
Remove .codeclimate.yml
2017-10-26 09:03:11 +02:00
Julius Volz 554ccf551c Remove .codeclimate.yml
See https://github.com/prometheus/prometheus/pull/3333#issuecomment-339209982
2017-10-26 04:51:11 +00:00
Paul Gier c4c3205d76 storage/tsdb: check that max block duration is larger than min
If the user accidentally sets the max block duration smaller than the min,
the current error is not informative.  This change just performs the check
earlier and improves the error message.
2017-10-25 19:24:49 -05:00
Sunny Klair 4fdf9b195c Validate index TOC checksum on read 2017-10-25 18:12:13 -04:00
Fabian Reinartz ce63a5a855 Merge pull request #3352 from prometheus/rc2
Cut v2.0.0-rc.2
2017-10-25 20:39:39 +02:00
Oren Shomron 6ca5e52b69 Typo in prometheus_tsdb_head_samples_appended_total description (#188) 2017-10-25 19:12:18 +01:00
Thibault Chataigner fc4406201e Tsdb StartTime : Use a simplier way to compute StartTime 2017-10-25 17:41:00 +02:00
Fabian Reinartz 013f368449 *: cut v2.0.0-rc.2 2017-10-25 17:02:59 +02:00
Fabian Reinartz b5d1ec53da vendor: update prometheus/tsdb 2017-10-25 16:54:39 +02:00
Fabian Reinartz 5d28c849c7 Merge pull request #187 from prometheus/cutchunk
Ensure near-empty chunks end at correct boundary
2017-10-25 16:52:11 +02:00
Jason Anderson 808f79f00a Feature: Allow getting credentials via EC2 role (#3343)
* Allow getting credentials via EC2 role

This is subtly different than the existing `role_arn` solution, which
allows Prometheus to assume an IAM role given some set of credentials
already in-scope. With EC2 roles, one specifies the role at instance
launch time (via an instance profile.) The instance then exposes
temporary credentials via its metadata. The AWS Go SDK exposes a
credential provider that polls the [instance metadata endpoint][1]
already, so we can simply use that and it will take care of renewing the
credentials when they expire.

Without this, if this is being used inside EC2, it is difficult to
cleanly allow the use of STS credentials. One has to set up a proxy role
that can assume the role you really want, and launch the EC2 instance
with the proxy role. This isn't very clean, and also doesn't seem to be
[supported very well][2].

[1]:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
[2]: https://github.com/aws/aws-cli/issues/1390

* Automatically try to detect EC2 role credentials

The `Available()` function exposed on ec2metadata returns a simple
true/false if the ec2 metadata is available. This is the best way to
know if we're actually running in EC2 (which is the only valid use-case
for this credential provider.)

This allows this to "just work" if you are using EC2 instance roles.
2017-10-25 14:15:39 +01:00
Fabian Reinartz 82796db37b Ensure near-empty chunks end at correct boundary
We were determining a chunk's end time once it was one quarter full to
compute it so all chunks have uniform number of samples.
This accidentally skipped the case where series started near the end of
a chunk range/block and never reached that threshold. As a result they
got persisted but were continued across the range.

This resulted in corrupted persisted data.
2017-10-25 09:51:55 +02:00
Fabian Reinartz d109149d17 Merge pull request #186 from prometheus/closeallblocks
Ensure readers are closed on setup failure.
2017-10-25 09:31:06 +02:00
Fabian Reinartz c93162c751 Merge pull request #185 from prometheus/walslice
Limit WAL sample processing batch size
2017-10-25 09:30:52 +02:00
Fabian Reinartz 820bd8b170 Merge pull request #181 from prometheus/gcchunk
Fix dangling chunk reference panic
2017-10-25 09:30:16 +02:00
Fabian Reinartz 6ecdaa5314 Merge pull request #183 from prometheus/walbrokensegment
Truncate segments on broken header
2017-10-25 09:27:44 +02:00
Julius Volz 099df0c5f0 Migrate "golang.org/x/net/context" -> "context" (#3333)
In some places, where ctxhttp or gRPC are concerned, we still need to use the
old contexts.
2017-10-24 21:21:42 -07:00
Fabian Reinartz 7bc07d80b6 Merge pull request #180 from prometheus/fixbugz
Fix race in symbol table re-creation
2017-10-24 11:39:13 +02:00
Takashi Kokubun bd52950ce8 Fix invalid syntax in graph.css (#3309) 2017-10-24 10:48:58 +02:00
Tom Wilkie 4bbef0ec30 Bypass the fanout storage merging if no remote storage is configured. 2017-10-23 21:34:53 +01:00
Fabian Reinartz a57ea79660 Close index reader properly 2017-10-23 21:59:18 +02:00
Fabian Reinartz f8e88bfdb7 Close previous block queriers on error
This ensures we close all previously opened queriers if on of the block
querier fails to open.
Also swap in new blocks before closing old ones to avoid the situation
in general. Make read locking of blocks more conservative to avoid
unnecessary retries by clients, e.g. when blocks are getting closed
before we can successfully instantiate querier against them.
2017-10-23 21:56:12 +02:00
Fabian Reinartz 9749aa2a3e head: limit WAL sample processing batch size 2017-10-23 16:22:24 +02:00