mirror of
https://github.com/prometheus/prometheus.git
synced 2024-11-10 07:34:04 -08:00
0e202dacb4
This will fix issue #1035 and will also help to make issue #1264 less bad. The fundamental problem in the current code: In the preload phase, we quite accurately determine which chunks will be used for the query being executed. However, in the subsequent step of creating series iterators, the created iterators are referencing _all_ in-memory chunks in their series, even the un-pinned ones. In iterator creation, we copy a pointer to each in-memory chunk of a series into the iterator. While this creates a certain amount of allocation churn, the worst thing about it is that copying the chunk pointer out of the chunkDesc requires a mutex acquisition. (Remember that the iterator will also reference un-pinned chunks, so we need to acquire the mutex to protect against concurrent eviction.) The worst case happens if a series doesn't even contain any relevant samples for the query time range. We notice that during preloading but then we will still create a series iterator for it. But even for series that do contain relevant samples, the overhead is quite bad for instant queries that retrieve a single sample from each series, but still go through all the effort of series iterator creation. All of that is particularly bad if a series has many in-memory chunks. This commit addresses the problem from two sides: First, it merges preloading and iterator creation into one step, i.e. the preload call returns an iterator for exactly the preloaded chunks. Second, the required mutex acquisition in chunkDesc has been greatly reduced. That was enabled by a side effect of the first step, which is that the iterator is only referencing pinned chunks, so there is no risk of concurrent eviction anymore, and chunks can be accessed without mutex acquisition. To simplify the code changes for the above, the long-planned change of ValueAtTime to ValueAtOrBefore time was performed at the same time. (It should have been done first, but it kind of accidentally happened while I was in the middle of writing the series iterator changes. Sorry for that.) So far, we actively filtered the up to two values that were returned by ValueAtTime, i.e. we invested work to retrieve up to two values, and then we invested more work to throw one of them away. The SeriesIterator.BoundaryValues method can be removed once #1401 is fixed. But I really didn't want to load even more changes into this PR. Benchmarks: The BenchmarkFuzz.* benchmarks run 83% faster (i.e. about six times faster) and allocate 95% fewer bytes. The reason for that is that the benchmark reads one sample after another from the time series and creates a new series iterator for each sample read. To find out how much these improvements matter in practice, I have mirrored a beefy Prometheus server at SoundCloud that suffers from both issues #1035 and #1264. To reach steady state that would be comparable, the server needs to run for 15d. So far, it has run for 1d. The test server currently has only half as many memory time series and 60% of the memory chunks the main server has. The 90th percentile rule evaluation cycle time is ~11s on the main server and only ~3s on the test server. However, these numbers might get much closer over time. In addition to performance improvements, this commit removes about 150 LOC. |
||
|---|---|---|
| cmd | ||
| config | ||
| console_libraries | ||
| consoles | ||
| documentation | ||
| notification | ||
| promql | ||
| retrieval | ||
| rules | ||
| scripts | ||
| storage | ||
| template | ||
| util | ||
| vendor | ||
| version | ||
| web | ||
| .dockerignore | ||
| .gitignore | ||
| .travis.yml | ||
| AUTHORS.md | ||
| CHANGELOG.md | ||
| circle.yml | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| LICENSE | ||
| Makefile | ||
| NOTICE | ||
| README.md | ||
Prometheus 
Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.
Prometheus' main distinguishing features as compared to other monitoring systems are:
- a multi-dimensional data model (timeseries defined by metric name and set of key/value dimensions)
- a flexible query language to leverage this dimensionality
- no dependency on distributed storage; single server nodes are autonomous
- timeseries collection happens via a pull model over HTTP
- pushing timeseries is supported via an intermediary gateway
- targets are discovered via service discovery or static configuration
- multiple modes of graphing and dashboarding support
- support for hierarchical and horizontal federation
Architecture overview
Install
There are various ways of installing Prometheus.
Precompiled binaries
Precompiled binaries for released versions are available in the releases section of the GitHub repository. Using the latest production release binary is the recommended way of installing Prometheus.
Debian and RPM packages are being worked on.
Building from source
To build Prometheus from the source code yourself you need to have a working Go environment with version 1.5 or greater installed.
You can directly use the go tool to download and install the prometheus
and promtool binaries into your GOPATH. We use Go 1.5's experimental
vendoring feature, so you will also need to set the GO15VENDOREXPERIMENT=1
environment variable in this case:
$ GO15VENDOREXPERIMENT=1 go get github.com/prometheus/prometheus/cmd/...
$ prometheus -config.file=your_config.yml
You can also clone the repository yourself and build using make:
$ mkdir -p $GOPATH/src/github.com/prometheus
$ cd $GOPATH/src/github.com/prometheus
$ git clone https://github.com/prometheus/prometheus.git
$ cd prometheus
$ make build
$ ./prometheus -config.file=your_config.yml
The Makefile provides several targets:
- build: build the
prometheusandpromtoolbinaries - test: run the tests
- format: format the source code
- vet: check the source code for common errors
- assets: rebuild the static assets
- docker: build a docker container for the current
HEAD
More information
- The source code is periodically indexed: Prometheus Core.
- You will find a Travis CI configuration in
.travis.yml. - All of the core developers are accessible via the Prometheus Developers Mailinglist and the
#prometheuschannel onirc.freenode.net.
Contributing
Refer to CONTRIBUTING.md
License
Apache License 2.0, see LICENSE.