Commit graph

205 commits

Author SHA1 Message Date
Julius Volz f92532f254 api: Consolidate web API contexts
This is based on the common/route changes in
https://github.com/prometheus/common/pull/61.
2016-09-21 03:22:20 +02:00
Julius Volz c187308366 storage: Contextify storage interfaces.
This is based on https://github.com/prometheus/prometheus/pull/1997.

This adds contexts to the relevant Storage methods and already passes
PromQL's new per-query context into the storage's query methods.
The immediate motivation supporting multi-tenancy in Frankenstein, but
this could also be used by Prometheus's normal local storage to support
cancellations and timeouts at some point.
2016-09-19 16:29:07 +02:00
Julius Volz ed5a0f0abe promql: Allow per-query contexts.
For Weaveworks' Frankenstein, we need to support multitenancy. In
Frankenstein, we initially solved this without modifying the promql
package at all: we constructed a new promql.Engine for every
query and injected a storage implementation into that engine which would
be primed to only collect data for a given user.

This is problematic to upstream, however. Prometheus assumes that there
is only one engine: the query concurrency gate is part of the engine,
and the engine contains one central cancellable context to shut down all
queries. Also, creating a new engine for every query seems like overkill.

Thus, we want to be able to pass per-query contexts into a single engine.

This change gets rid of the promql.Engine's built-in base context and
allows passing in a per-query context instead. Central cancellation of
all queries is still possible by deriving all passed-in contexts from
one central one, but this is now the responsibility of the caller. The
central query context is now created in main() and passed into the
relevant components (web handler / API, rule manager).

In a next step, the per-query context would have to be passed to the
storage implementation, so that the storage can implement multi-tenancy
or other features based on the contextual information.
2016-09-19 15:38:17 +02:00
beorn7 39c4915401 federation: Collapse time series of the same name
This will avoid duplicate MetricFamilies, thereby shrinking the size
of the federation payload and also creating legal text format.

Also, add unit tests for federation. They were also needed for the
previous state of the code, but were missing.
2016-09-14 19:35:20 +02:00
Dmitry Vorobev 273e457da4 web: return status code and error message for config resource 2016-07-15 10:15:24 +02:00
Fabian Reinartz 59d26e8536 web: add -web.route-prefix flag
Fixes #1191
2016-07-07 11:49:16 +02:00
Fabian Reinartz 97f62c021e api: drop legacy API 2016-06-23 17:14:32 +02:00
Fabian Reinartz 25135c3691 web: round last scrape timestamp to milliseconds 2016-06-08 20:23:54 +02:00
Ali Reza 624717e801 Log HTTP server errors using common/log
change naming to be more descriptive.
Fixes #1651

remove pointer receiver

use common/log error logger
2016-05-27 00:20:40 +07:00
beorn7 da8cb10b43 Partition the status tab into items in a dropdown
I got feedback from different sources about rules and targets being
too heavy in the status tab if their are lots of them.

This change also allows for more fine-granular locking.
2016-05-18 18:13:55 +02:00
Steve Durrheimer 399d5c6375
Make version informations consistent between prometheus components 2016-05-05 22:33:18 +02:00
Fabian Reinartz f2e359962c Sort exported targets 2016-03-08 17:12:27 +01:00
Tobias Schmidt 50079a85a1 Make time parameter optional in v1 query API
If no time paramter is provided, the current server timestamp is used.
2015-11-11 13:30:06 -08:00
Fabian Reinartz a542cc8609 Remove -web.use-local-assets 2015-11-11 17:58:03 +01:00
Fabian Reinartz e3b6ec9784 Switch to common/log 2015-10-03 10:21:43 +02:00
Julius Volz dac26cef71 Rename global "labels" config option to "external_labels". 2015-09-29 20:54:20 +02:00
Fabian Reinartz 398bbf906b Switch to common/route package 2015-09-24 17:08:47 +02:00
Julius Volz 0179bbfe03 Fix / -> /graph redirect when using a path prefix. 2015-09-23 12:36:18 +02:00
Fabian Reinartz 171f50706a Fix unkeyed field errors. 2015-09-18 17:00:08 +02:00
Fabian Reinartz d85cee0277 Remove compression from web interface 2015-09-17 16:33:14 +02:00
Fabian Reinartz f8a25f6af7 Apply HTTP handler compression everywhere 2015-09-17 14:49:50 +02:00
Fabian Reinartz 896928ad00 Fix redirect path 2015-09-12 09:02:34 +02:00
Fabian Reinartz d7b21f2fdb Merge pull request #1039 from prometheus/startpage
Change UI start page
2015-09-12 08:44:54 +02:00
Fabian Reinartz 87b1d47e2b Change UI start page 2015-09-12 08:40:46 +02:00
Fabian Reinartz 9bbd9264e2 Add global labels to federation 2015-09-03 16:38:03 +02:00
Julius Volz da2499d5a5 Add POST handler for /debug to fix CPU profiling.
`go tool pprof` sends a POST request to enable CPU profiling, so we also
need to handle that method.
2015-08-26 17:42:25 +02:00
Julius Volz 995d3b831d Fix most golint warnings.
This is with `golint -min_confidence=0.5`.

I left several lint warnings untouched because they were either
incorrect or I felt it was better not to change them at the moment.
2015-08-26 12:44:46 +02:00
Fabian Reinartz 306e8468a0 Switch from client_golang/model to common/model 2015-08-21 13:33:38 +02:00
Julius Volz 274e9d6955 Exit when web server encounters a startup error 2015-08-20 18:23:57 +02:00
Jan Berktold fa929a8345 Change /reload to /-/reload & and fix channel type 2015-08-11 12:26:41 +02:00
Jan Berktold 2bf7048dbb Add reload handler to web 2015-08-11 11:27:15 +02:00
Julius Volz c5c7f92960 Fix /debug/pprof endpoint with new web router.
This got broken in
78047326b4
since it stopped using the DefaultServeMux.

This approach will defer pprof requests to the DefaultServeMux, which
may or may not have pprof enabled (in Prometheus, it gets it included in
main.go). An alternative approach would be to duplicate the four lines in
https://golang.org/src/net/http/pprof/pprof.go#L62. When choosing that
approach though, we would not automatically gain any new endpoints added
by net/http/pprof or other /debug endpoints in the future.
2015-07-22 12:28:46 +02:00
Brian Brazil 7b5a29fc8f web: Fix regression in .Path
.Path is documented as removing /consoles/,
recent changes added in a leading / which broke
the provided console templates menu system.
2015-07-10 10:31:35 +01:00
Julius Volz fcff35b43e Consolidate external reachability flags into one.
Besides fixing https://github.com/prometheus/prometheus/issues/805 by
making the entire externally reachable server URL configurable, this
adds tests for the "globalURL" template function and makes it easier to
test other such functions in the future.

This breaks the `web.Hostname` flag (and introduces `web.external-url`).
This flag is likely only used by few users, so I hope that's
justifiable.

Fixes https://github.com/prometheus/prometheus/issues/805
2015-07-03 13:39:10 +02:00
Julius Volz d7ea0397d4 Use templates.TemplateExpander for all page templates.
This will allow us to use all of our shiny templating functions in the
main page templates as well.
2015-06-23 17:53:06 +02:00
Fabian Reinartz 53b9d5917d web: improve target URL handling and display. 2015-06-23 13:45:15 +02:00
Fabian Reinartz 39a8254963 web: add basic federation support.
This commit adds a federation handler on /federate. It accepts `match[]`
query parameters containing vector selectors. Their intersection determines
the in-memory metrics that are returned in the same way as the
/metrics endpoint does (modulo sorting).
2015-06-23 13:45:15 +02:00
Fabian Reinartz 119801027f web: expose a version information endpoint 2015-06-17 14:49:31 +02:00
Fabian Reinartz 39edc2df7a version: move version information into separate package.
Version information is determined at build-time and thus there is
no need to pass it down from main. In its own package it can
be used from various other packages.
2015-06-16 14:48:29 +02:00
Fabian Reinartz 85d7c7640a web: remove flags, refactor handlers 2015-06-15 19:01:06 +02:00
Sindre Myren a081c9436b Building without a Makefile (on Windows)
Main changes:

- Switched to using `go-bindata` in place of `scripts/embed-static.sh`.
- Support for building Prometheus without a `Makefile`.
- Minor typo fix to make Prometheus build on Windows (without Makefiles).

Please note that this does not mean that prometheus will work on Windows.
There are still failing tests!
2015-06-10 20:43:59 +02:00
Fabian Reinartz 5b713911e3 web/api: enable running API legacy and v1 in parallel 2015-06-08 19:11:48 +02:00
Fabian Reinartz e88e5f680b web: simplify prefix handling using util/route package. 2015-06-03 15:53:04 +02:00
Fabian Reinartz 78047326b4 web: cleanup initialization of web service. 2015-06-03 08:45:43 +02:00
Julius Volz 09fcf505f5 Fix slash position in MustBuildServerURL(). 2015-06-01 12:54:19 +02:00
Julius Volz d7c015c149 Convert pathPrefix to not have trailing slash. 2015-06-01 12:43:17 +02:00
Julius Volz 837ffcea29 Merge pull request #726 from prometheus/improve-redirections
Improve web redirection and 404 behavior.
2015-05-23 16:16:26 +02:00
Tobias Gesellchen 84d42f48e5 allow to override the default os.Hostname() when creating PrometheusURLs 2015-05-23 12:59:38 +02:00
Julius Volz fcd33bbfd8 Improve web redirection and 404 behavior.
Previously we redirected any non-existent path to the root (or path
prefix).

The new behavior:

With no path prefix:

- "" -> "/"
- "/biz" -> 404

With path prefix of "/foo/bar":

- "" -> "/foo/bar/"
- "/" -> "/foo/bar/"
- "/foo/bar" -> "/foo/bar/"
- "/biz" -> /foo/bar/biz"
  (anything not starting with the path prefix gets the prefix prepended)
- "/foo/bar/biz" -> 404
2015-05-23 12:46:16 +02:00
Julius Volz 267fd34156 Switch Prometheus to use github.com/prometheus/log.
This change is conceptually very simple, although the diff is large. It
switches logging from "github.com/golang/glog" to
"github.com/prometheus/log", while not actually changing any log
messages. V(1)-style logging has been changed to be log.Debug*().
2015-05-20 18:19:32 +02:00
Fabian Reinartz a216b1cb27 Merge pull request #704 from prometheus/fabxc/cfg_check
Clean storage shutdown on startup errors, require initial config
2015-05-19 16:50:05 +02:00
Fabian Reinartz 5d3024fd3e Restructure component initialization 2015-05-19 14:41:47 +02:00
Fabian Reinartz 385919a65a Avoid inter-component blocking if ingestion/scraping blocks.
Appending to the storage can block for a long time. Timing out
scrapes can also cause longer blocks. This commit avoids that those
blocks affect other compnents than the target itself.
Also the Target interface was removed.
2015-05-18 17:58:51 +02:00
Fabian Reinartz 1a2d57b45c Move template functionality out of target.
The target implementation and interface contain methods only serving a
specific purpose of the templates. They were moved to the template
as they operate on more fundamental target data.
2015-05-18 13:35:43 +02:00
Brian Brazil c3a2b63fe9 Add stddev and stdvar aggregation functions.
This adds the population standard deviation and
variance as aggregation functions, useful for
spotting how many standard deviations some samples
are from the mean.
2015-04-17 00:30:03 +01:00
Ceesjan Luiten 6f5f33bd4d #625 Fix incorrect consoles URL 2015-04-06 17:30:13 +02:00
Ceesjan Luiten 0e18784c64 Make all paths absolute to support proxies 2015-04-02 20:36:47 +02:00
beorn7 be11cb2b07 Remove the sample ingestion channel.
The one central sample ingestion channel has caused a variety of
trouble. This commit removes it. Targets and rule evaluation call an
Append method directly now. To incorporate multiple storage backends
(like OpenTSDB), storage.Tee forks the Append into two different
appenders.

Note that the tsdb queue manager had its own queue anyway. It was a
queue after a queue... Much queue, so overhead...

Targets have their own little buffer (implemented as a channel) to
avoid stalling during an http scrape. But a new scrape will only be
started once the old one is fully ingested.

The contraption of three pipelined ingesters was removed. A Target is
an ingester itself now. Despite more logic in Target, things should be
less confusing now.

Also, remove lint and vet warnings in ast.go.
2015-03-15 14:08:22 +01:00
Julius Volz 989bc86bcb Make flag names consistent across projects. 2015-02-08 23:29:57 +01:00
Bjoern Rabenstein 5859b74f1b Clean up license issues.
- Move CONTRIBUTORS.md to the more common AUTHORS.
- Added the required NOTICE file.
- Changed "Prometheus Team" to "The Prometheus Authors".
- Reverted the erroneous changes to the Apache License.
2015-01-21 20:07:45 +01:00
Bjoern Rabenstein b1e4956142 Apply a giant code cleanup.
Essentially:

- Remove unused code.

- Make it 'go vet' clean. The only remaining warnings are in generated code.

- Make it 'golint' clean. The only remaining warnings are in gerenated code.

- Smoothed out same minor things.

Change-Id: I3fe5c1fbead27b0e7a9c247fee2f5a45bc2d42c6
2014-12-10 16:16:49 +01:00
Bjoern Rabenstein 14bda4180c Changes after pair code review.
Change-Id: Ib72d40f8e9027818cfbbd32a7a7201eebda07455
2014-11-25 17:12:59 +01:00
Brian Brazil fd34e4061d Add back consoles link.
Goes in index.html in consoles or else user data, if present.

Change-Id: I5303d30aa24ca0c20d2e0f49121e04a260b9c4f4
2014-11-25 17:08:26 +01:00
Julius Volz e7ed39c9a6 Initial experimental snapshot of next-gen storage.
Change-Id: Ifb8709960dbedd1d9f5efd88cdd359ee9fa9d26d
2014-11-25 17:02:00 +01:00
Brian Brazil 4a2b96f848 Remove backoff on scrape failure.
Having metrics with variable timestamps inconsistently
spaced when things fail will make it harder to write correct rules.

Update status page, requires some refactoring to insert a function.

Change-Id: Ie1c586cca53b8f3b318af8c21c418873063738a8
2014-11-25 17:02:00 +01:00
Brian Brazil eb5d928da7 Fix console handler.
This was accidnetally broken in 2128d9d811.

Change-Id: I50ea1fdb8ae4d28ae4555410bee97e5037692aa5
2014-11-25 17:01:59 +01:00
Bjoern Rabenstein 8956faeccb Migrate to new client_golang.
This change will only be submitted when the new client_golang has been
moved to the new version.

Change-Id: Ifceb59333072a08286a8ac910709a8ba2e3a1581
2014-11-25 17:01:59 +01:00
Brian Brazil e27447da5c Remove the broken "User Dashboard" link.
Due to the lack of a </a>, this makes the entire header render badly.
Accordingly it's safe to assume noone is using it, so remove it.
With the new console template support, we'll need to something a bit
more nuanced later.

Change-Id: I3424bed6aea18cbd4c63ad48f98808098dadc3ad
2014-11-25 17:01:59 +01:00
Brian Brazil e041c0cd46 Add console and alert templates with access to all data.
Move rulemanager to it's own package to break cicrular dependency.
Make NewTestTieredStorage available to tests, remove duplication.

Change-Id: I33b321245a44aa727bfc3614a7c9ae5005b34e03
2014-05-30 16:24:56 +01:00
Matt T. Proud 2064f32662 Clean up quitting behavior and add quit trigger.
The closing of Prometheus now using a sync.Once wrapper to prevent
any accidental multiple invocations of it, which could trigger
corruption or a race condition.  The shutdown process is made more
verbose through logging.

A not-enabled by default web handler has been provided to trigger a
remote shutdown if requested for debugging purposes.

Change-Id: If4fee75196bbff1fb1e4a4ef7e1cfa53fef88f2e
2014-04-15 21:40:04 +02:00
Conor Hennessy eba01d1119 Remove usage of gorest.
Due to on going issues, we've decided to remove gorest. It started with gorest
not being thread-safe (it does introspection to create a new handler which is
an easy process to mess up with multiple threads of execution):
    https://code.google.com/p/gorest/issues/detail?id=15
While the issue has been marked fixed, it looks like the patch has introduced
more problems than the original issue and simply doesn't work properly.
I'm not sure the behaviour was thought through properly. If a new instance is
needed every request then a handler-factory is needed or the library needs to
set expectations about how the new objects should interact with their
constructor state.
While it was tempting to try out another routing library, I think for now
it's better to use dumb vanilla Go routing. At least until we decide which
URL format we intend to standardize on.

Change-Id: Ica3da135d05f8ab8fc206f51eeca4f684f8efa0e
2013-10-23 14:19:14 +02:00
Matt T. Proud 4a87c002e8 Update low-level i'faces to reflect wireformats.
This commit fixes a critique of the old storage API design, whereby
the input parameters were always as raw bytes and never Protocol
Buffer messages that encapsulated the data, meaning every place a
read or mutation was conducted needed to manually perform said
translations on its own.  This is taxing.

Change-Id: I4786938d0d207cefb7782bd2bd96a517eead186f
2013-09-04 17:13:58 +02:00
Matt T. Proud 972e856d9b Kill the curation state channel.
The use of the channels for curation state were always unidiomatic.

Change-Id: I1cb1d7175ebfb4faf28dff84201066278d6a0d92
2013-08-13 17:20:22 +02:00
Julius Volz 0003027dce Add needed trailing spaces in logs. 2013-08-12 18:22:48 +02:00
Julius Volz aa5d251f8d Use github.com/golang/glog for all logging. 2013-08-12 17:54:36 +02:00
Julius Volz ecf0ee8f39 Transfer alerting rule and Prometheus URL to alertmanager. 2013-08-09 18:32:13 +02:00
Julius Volz 1aa8f071b9 Add content compression support to API HTTP responses. 2013-06-28 16:56:44 +02:00
Julius Volz 0226d1ac7a Implement alerts dashboard and expression console links. 2013-06-13 22:35:40 +02:00
Matt T. Proud a73f061d3c Persist solely Protocol Buffers.
An design question was open for me in the beginning was whether to
serialize other types to disk, but Protocol Buffers quickly won out,
which allows us to drop support for other types.  This is a good
start to cleaning up a lot of cruft in the storage stack and
can let us eventually decouple the various moving parts into
separate subsystems for easier reasoning.

This commit is not strictly required, but it is a start to making
the rest a lot more enjoyable to interact with.
2013-06-08 11:02:35 +02:00
Bernerd Schaefer f7a2436665 Include link to user dashboard when provided 2013-06-07 11:17:17 +02:00
Bernerd Schaefer 1d794896ac Support user-provided static asset directory
[fix #159]
2013-06-07 10:25:12 +02:00
Matt T. Proud b224251981 Simplify compaction and expose database sizes.
This commit simplifies the way that compactions across a database's
keyspace occur due to reading the LevelDB internals. Secondarily it
introduces the database size estimation mechanisms.

Include database health and help interfaces.

Add database statistics; remove status goroutines.

This commit kills the use of Go routines to expose status throughout
the web components of Prometheus. It also dumps raw LevelDB status
on a separate /databases endpoint.
2013-05-14 12:29:53 +02:00
Bernerd Schaefer 9183302b1f Web handler returns 404 for favicon requests 2013-05-07 18:15:17 +02:00
Matt T. Proud 3b9b1c6ab4 Define dependencies for web. stack concretely.
This commit destroys the use of AppState, which makes passing
concrete state along to various serving components onerous.
2013-05-06 11:13:12 +02:00
Johannes 'fish' Ziemke ba289ef7cd Add the console to the main/graph ui. 2013-05-02 12:19:34 +02:00
Matt T. Proud 3362bf36e2 Include curator status in web heads-up-display. 2013-04-29 12:40:33 +02:00
Matt T. Proud a48ab34dd0 Refresh Prometheus client API usage.
The client API has been updated per https://github.com/prometheus/client_golang/pull/9.
2013-04-28 19:40:30 +02:00
Bernerd Schaefer 862054e88b web.StartServing prints listening address 2013-04-25 11:59:39 +02:00
Julius Volz a2623efcdf Register pprof /debug endpoints with custom HTTP mux. 2013-04-22 13:21:24 +02:00
Johannes 'fish' Ziemke 0f775a4178 Use Sprintf instead of string concatenation. 2013-04-05 13:41:52 +02:00
Johannes 'fish' Ziemke 1c091a9723 Use client_golang.exp for automatic telemetry. 2013-04-05 13:24:50 +02:00
Johannes 'fish' Ziemke 3aead8468c Fix typo. 2013-04-04 19:19:58 +02:00
Johannes 'fish' Ziemke 84d17b4d97 Replace YieldExporter() with Handler() 2013-04-04 19:08:20 +02:00
Johannes 'fish' Ziemke 1a3fb459e0 Add navigation menu to all pages + change routes.
To achieve that, this PR
- converts static/index.html ("console") and graph to templates
- moved the handlebars template to separated file to avoid escaping issues

Route changes:
/status -> /
/static -> /console
/static/graph.html -> /graph
2013-04-04 16:36:53 +02:00
Johannes 'fish' Ziemke f27adac848 Split status template into two templates.
This is now using a base template (_base.html) for the 'layout' of the web UI.
Within that base template, the actual content templates get rendered.
2013-03-27 17:40:01 +01:00
Johannes 'fish' Ziemke b290c821d1 Renamed "localAssets" cmd flag to "useLocalAssets". 2013-03-25 12:09:39 +01:00
Matt T. Proud 1b0ca377af Reformat. 2013-03-21 18:11:34 +01:00
Johannes 'fish' Ziemke a5e18b0229 Rename localAssets to useLocalAssets. 2013-03-21 15:55:48 +01:00
Johannes 'fish' Ziemke bf5d312a74 Add flag to read assets from local files. 2013-03-21 13:55:59 +01:00
Johannes 'fish' Ziemke fc16580b4c Ship assets in compiled binary.
- utility/embed-static.sh, get called in Makefile to create go map from files
- web/blob/blob.go implements http Handle for serving the files from the map
- web/status.go uses blog.GetFile() to get the template file

The assets are gzipped and decompressed on demand.
2013-03-19 17:40:27 +01:00
Julius Volz f1fc7d717a Allow replacing job targets via HTTP API.
This roughly comprises the following changes:

- index target pools by job instead of scrape interval
- make targets within a pool exchangable while preserving existing
  health state for targets
- allow exchanging targets via HTTP API (PUT)
- show target lists in /status (experimental, for own debug use)
2013-02-28 21:33:29 +01:00
Julius Volz 23374788d3 Beginnings of a Prometheus status page. 2013-02-14 19:03:17 +01:00
Matt T. Proud d1e7f5bc4b The Prometheus Go client has a new handler API. 2013-02-13 09:46:41 -08:00
Julius Volz 2859227fdb Make HTTP listen address a flag. 2013-02-08 15:42:09 +01:00
Julius Volz 0cbd03ccf9 Move web-related code/resources to a subdirectory. 2013-02-08 14:52:36 +01:00