Commit graph

3351 commits

Author SHA1 Message Date
Ingo Gottwald 3b546d061f Add support for GCE discovery 2016-09-16 08:55:33 +02:00
Ingo Gottwald fefcd6eef2 Add deps for google cloud support 2016-09-16 08:51:58 +02:00
Tobias Schmidt 4b850970a2 Merge pull request #1999 from prometheus/print-promu
Add promu installation logging to Makefile
2016-09-15 19:12:15 -04:00
Julius Volz 099adab253 Merge pull request #1987 from tomwilkie/1982-die-grpc-die
Switch back to protos over HTTP, instead of grpc.
2016-09-16 01:01:47 +02:00
Julius Volz 92d60ba4c0 Add promu installation logging to Makefile
Due to bad GitHub connectivity, "make" frequently got stuck at the promu
step for me, and I was thinking that "format" was taking a long time
because the promu step wasn't logged. All other Makefile targets have
log statements...
2016-09-16 00:59:56 +02:00
Tom Wilkie d83879210c Switch back to protos over HTTP, instead of GRPC.
My aim is to support the new grpc generic write path in Frankenstein.  On the surface this seems easy - however I've hit a number of problems that make me think it might be better to not use grpc just yet.

The explanation of the problems requires a little background.  At weave, traffic to frankenstein need to go through a couple of services first, for SSL and to be authenticated.  So traffic goes:

    internet -> frontend -> authfe -> frankenstein

- The frontend is Nginx, and adds/removes SSL.  Its done this way for legacy reasons, so the certs can be managed in one place, although eventually we imagine we'll merge it with authfe.  All traffic from frontend is sent to authfe.
- Authfe checks the auth tokens / cookie etc and then picks the service to forward the RPC to.
- Frankenstein accepts the reads and does the right thing with them.

First problem I hit was Nginx won't proxy http2 requests - it can accept them, but all calls downstream are http1 (see https://trac.nginx.org/nginx/ticket/923).  This wasn't such a big deal, so it now looks like:

    internet --(grpc/http2)--> frontend --(grpc/http1)--> authfe --(grpc/http1)--> frankenstein

Next problem was golang grpc server won't accept http1 requests (see https://groups.google.com/forum/#!topic/grpc-io/JnjCYGPMUms).  It is possible to link a grpc server in with a normal go http mux, as long as the mux server is serving over SSL, as the golang http client & server won't do http2 over anything other than an SSL connection.  This would require making all our service to service comms SSL.  So I had a go a writing a grpc http1 server, and got pretty far.  But is was a bit of a mess.

So finally I thought I'd make a separate grpc frontend for this, running in parallel with the frontend/authfe combo on a different port - and first up I'd need a grpc reverse proxy.  Ideally we'd have some nice, generic reverse proxy that only knew about a map from service names -> downstream service, and didn't need to decode & re-encode every request as it went through.  It seems like this can't be done with golang's grpc library - see https://github.com/mwitkow/grpc-proxy/issues/1.

And then I was surprised to find you can't do grpc from browsers! See http://www.grpc.io/faq/ - not important to us, but I'm starting to question why we decided to use grpc in the first place?

It would seem we could have most of the benefits of grpc with protos over HTTP, and this wouldn't preclude moving to grpc when its a bit more mature?  In fact, the grcp FAQ even admits as much:

> Why is gRPC better than any binary blob over HTTP/2?
> This is largely what gRPC is on the wire.
2016-09-15 23:21:54 +01:00
Tom Wilkie e0989fde89 Remove grpc vendoring. 2016-09-15 23:15:56 +01:00
Tom Wilkie bcd43e82c6 Add go_import_path to travis so it works on a fork. (#1995) 2016-09-15 17:05:56 -04:00
Tobias Schmidt 3eef95962b Merge pull request #1965 from prometheus/beorn7/federation
federation: Collapse time series of the same name
2016-09-15 17:03:13 -04:00
beorn7 717dd8adac web: add more federation test scenarios 2016-09-15 15:23:55 +02:00
beorn7 784a8ad7c5 web: Inline httptest.NewRequest because it only exists in Go1.7+ 2016-09-15 15:06:36 +02:00
Fabian Reinartz 737ae60cea Merge pull request #1993 from prometheus/grobie/include-go-report
Link to goreport from README
2016-09-15 08:00:14 +02:00
Fabian Reinartz 4f7e6e8bf0 Merge pull request #1994 from prometheus/grobie/fix-small-issues
Fix low hanging code issues
2016-09-15 07:57:55 +02:00
Tobias Schmidt 29ced0090f Fix common english misspellings 2016-09-14 23:23:28 -04:00
Tobias Schmidt e2c12dcdb5 Add missing error check in persistence test 2016-09-14 23:16:47 -04:00
Tobias Schmidt 27074863b4 Print url.URLs correctly in tests 2016-09-14 23:15:18 -04:00
Tobias Schmidt 8f3b62bfe4 Simplify struct initialization 2016-09-14 23:13:27 -04:00
Tobias Schmidt b41a240c36 Link to goreport from README 2016-09-14 23:09:26 -04:00
beorn7 39c4915401 federation: Collapse time series of the same name
This will avoid duplicate MetricFamilies, thereby shrinking the size
of the federation payload and also creating legal text format.

Also, add unit tests for federation. They were also needed for the
previous state of the code, but were missing.
2016-09-14 19:35:20 +02:00
Julius Volz b24e5d63bc Add noop local storage engine.
This adds a flag -storage.local.engine which allows turning off local
storage in Prometheus. Instead of adding if-conditions and nil checks to
all parts of Prometheus that deal with Prometheus's local storage
(including the web interface), disabling local storage simply means
replacing the normal local storage with a noop version that throws
samples away and returns empty query results. We also don't add the noop
storage to the fanout appender to decrease internal overhead.

Instead of returning empty results, an alternate behavior could be to
return errors on any query that point out that the local storage is
disabled. Not sure which one is more preferable, so I went with the
empty result option for now.
2016-09-14 13:18:05 +02:00
Tobias Schmidt 03855679a1 Merge pull request #1984 from mattbostock/1983_add_minute
PromQL: Add minute() function
2016-09-12 18:20:38 -04:00
Matt Bostock a0201036fa PromQL: Add tests for time/date funcs with arg
Add tests for the date and time functions where an argument is
specified.

Suggested by @grobie:
https://github.com/prometheus/prometheus/pull/1984#issuecomment-246508286

`1136239445` is the reference time used by Go:
https://golang.org/src/time/format.go
2016-09-12 23:12:43 +01:00
Matt Bostock 9628eb5998 PromQL: Add minute() function
Returns the minutes from the current time in UTC. Related to the
`hour()` function.

Fixes #1983.
2016-09-12 20:34:23 +01:00
Björn Rabenstein 3fd992d2d1 Merge pull request #1979 from prometheus/fabxc-patch-1-1
storage: fix Querier interface documentation
2016-09-12 11:39:25 +02:00
Fabian Reinartz 22296dcb85 storage: clarify sample/matcher relation in docs 2016-09-12 11:19:36 +02:00
Fabian Reinartz cc6f988a5e storage: fix Querier interface documentation 2016-09-12 10:48:54 +02:00
Julius Volz 815cb57af3 Merge pull request #1975 from prometheus/storage-test
storage: fix struct alignment issue in test
2016-09-11 04:26:37 +02:00
Fabian Reinartz 7bd7e63f97 storage: fix struct alignment issue in test
The uint64 `numCalls` ends up being not word-aligned on certain architectures,
which makes atomic reads/writes panic.
2016-09-11 00:32:57 +02:00
Björn Rabenstein 7e28b4509d Merge pull request #1960 from nghialv/master
Fix typo
2016-09-09 19:59:53 +02:00
Steve Durrheimer 1e26b96baa Merge pull request #1973 from prometheus/release-1.1
CircleCI improvement from release-1.1
2016-09-09 16:55:24 +02:00
Steve Durrheimer 07babe586e Merge pull request #1969 from prometheus/sdurrheimer-circle-use-base-image-for-tests
Use golang-builder base image for tests in CircleCI
2016-09-09 13:53:45 +02:00
Steve Durrheimer 0ebdd9c3e9
Use golang-builder base image for tests in CircleCI 2016-09-09 13:13:21 +02:00
Ulfsparre, Tommy 3730255392 remove deleted zookeeper nodes 2016-09-09 00:27:23 +02:00
Tobias Schmidt f805208c3e Merge pull request #1966 from dmilstein/make-test-not-hit-internet
Fix broken test which relied on DNS resolution #1962
2016-09-08 17:36:36 -04:00
Dan Milstein 0cb6b9962e Fix broken test which relied on DNS resolution #1962
Switched to testing by way of the static_configs rather than
dns_sd_config parameter.  Verified that the revised test both passes
without network access, and also still catches the bug it's supposed to
cover.
2016-09-08 16:59:46 -04:00
Björn Rabenstein e5a5dc172d Merge pull request #1964 from prometheus/release-1.1
Forward-merge the bug fix from release-1.1
2016-09-08 15:17:40 +02:00
Fabian Reinartz 36fbdcc30f Merge pull request #1961 from prometheus/beorn7/release
Cut 1.1.2
2016-09-08 15:04:07 +02:00
beorn7 03831e0e1e Cut 1.1.2 2016-09-08 14:17:49 +02:00
Björn Rabenstein 531991e4a1 Merge pull request #1958 from prometheus/grobie/fix-aggregator-label-error
Fix parsing of label names which are also keywords
2016-09-08 14:06:39 +02:00
nghialv 7655038184 fix typo 2016-09-08 19:01:32 +09:00
Tobias Schmidt 04ae6196f2 Fix parsing of label names which are also keywords
The current separation between lexer and parser is a bit fuzzy when it
comes to operators, aggregators and other keywords. The lexer already
tries to determine the type of a token, even though that type might
change depending on the context.

This led to the problematic behavior that no tokens known to the lexer
could be used as label names, including operators (and, by, ...),
aggregators (count, quantile, ...) or other keywords (for, offset, ...).

This change additionally checks whether an identifier is one of these
types. We might want to check whether the specific item identification
should be moved from the lexer to the parser.
2016-09-07 17:45:58 -04:00
Fabian Reinartz 07a3cd4851 Merge pull request #1950 from prometheus/readme
readme: direct link to Prometheus docker image
2016-09-07 14:44:42 +02:00
Fabian Reinartz 4bb2063047 readme: point to prometheus.io download section 2016-09-07 14:32:48 +02:00
Fabian Reinartz d48a1bedb1 Merge branch 'release-1.1' 2016-09-07 11:27:50 +02:00
Fabian Reinartz 24db241bd5 Merge pull request #1954 from prometheus/backport
Cut v1.1.1
2016-09-07 11:27:27 +02:00
Fabian Reinartz 333e2ed92d *: cut v1.1.1 2016-09-07 11:18:06 +02:00
Fabian Reinartz a2589e7815 retrieval: correctly handle IPv6 addresses
This updates all service discoveries to correctly
build the __address__ label for IPv6 addresses.
2016-09-07 11:13:53 +02:00
Fabian Reinartz 692ddc592c retrieval: fix IPv6 port default, add tests
This fixes port defaulting for IPv6 addresses and restructures
and test the construction of target label sets.
2016-09-07 11:13:53 +02:00
Fabian Reinartz f030cb5f01 retrieval: export NewHTTPClient 2016-09-07 11:13:53 +02:00
Fabian Reinartz fec3b54cfc Merge pull request #1946 from prometheus/ipv6
Fix IPv6 scraping
2016-09-06 17:18:28 +02:00