The Prometheus monitoring system and time series database.
Find a file
Brian Brazil 38d32e0686 Don't sort postings if we only have one block.
Sorting the heads postings can be quite slow.
We only need sorted series when merging with another
querier, so only sort then.
This will make big queries that only touch the head faster,
though queries that touch both the head and a block will still
be the same speed. This probably won't help much with graphing
unless the range is under an hour, however it should make most
recording rules faster.

Add gaurantee that remote read streaming produces sorted series.

PromQL benchmarks for histograms show only 2-3% improvement, but
they're only over 1k series.

benchmark                                                old ns/op      new ns/op      delta
BenchmarkQuerierSelect/Head/1of1000000-4                 1375486282     507657736      -63.09%
BenchmarkQuerierSelect/Head/10of1000000-4                1387859004     507769850      -63.41%
BenchmarkQuerierSelect/Head/100of1000000-4               1387087935     506029110      -63.52%
BenchmarkQuerierSelect/Head/1000of1000000-4              1386869064     504521986      -63.62%
BenchmarkQuerierSelect/Head/10000of1000000-4             1386213685     505210422      -63.55%
BenchmarkQuerierSelect/Head/100000of1000000-4            1392754988     529842406      -61.96%
BenchmarkQuerierSelect/Head/1000000of1000000-4           1569414722     725059506      -53.80%
BenchmarkQuerierSelect/SortedHead/1of1000000-4           1381019902     1370495863     -0.76%
BenchmarkQuerierSelect/SortedHead/10of1000000-4          1375696209     1366789468     -0.65%
BenchmarkQuerierSelect/SortedHead/100of1000000-4         1386009422     1364519297     -1.55%
BenchmarkQuerierSelect/SortedHead/1000of1000000-4        1377700532     1364486191     -0.96%
BenchmarkQuerierSelect/SortedHead/10000of1000000-4       1383539536     1369545314     -1.01%
BenchmarkQuerierSelect/SortedHead/100000of1000000-4      1410089163     1394731339     -1.09%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-4     1634744148     1581554956     -3.25%
BenchmarkQuerierSelect/Block/1of1000000-4                881741242      879839470      -0.22%
BenchmarkQuerierSelect/Block/10of1000000-4               880381562      882846038      +0.28%
BenchmarkQuerierSelect/Block/100of1000000-4              887519357      881016916      -0.73%
BenchmarkQuerierSelect/Block/1000of1000000-4             902194205      883433524      -2.08%
BenchmarkQuerierSelect/Block/10000of1000000-4            892321964      885130170      -0.81%
BenchmarkQuerierSelect/Block/100000of1000000-4           938604466      933527150      -0.54%
BenchmarkQuerierSelect/Block/1000000of1000000-4          1313510845     1295881124     -1.34%

benchmark                                                old allocs     new allocs     delta
BenchmarkQuerierSelect/Head/1of1000000-4                 4000056        4000018        -0.00%
BenchmarkQuerierSelect/Head/10of1000000-4                4000074        4000036        -0.00%
BenchmarkQuerierSelect/Head/100of1000000-4               4000254        4000216        -0.00%
BenchmarkQuerierSelect/Head/1000of1000000-4              4002054        4002016        -0.00%
BenchmarkQuerierSelect/Head/10000of1000000-4             4020054        4020016        -0.00%
BenchmarkQuerierSelect/Head/100000of1000000-4            4200054        4200016        -0.00%
BenchmarkQuerierSelect/Head/1000000of1000000-4           6000054        6000016        -0.00%
BenchmarkQuerierSelect/SortedHead/1of1000000-4           4000071        4000071        +0.00%
BenchmarkQuerierSelect/SortedHead/10of1000000-4          4000089        4000089        +0.00%
BenchmarkQuerierSelect/SortedHead/100of1000000-4         4000269        4000269        +0.00%
BenchmarkQuerierSelect/SortedHead/1000of1000000-4        4002069        4002069        +0.00%
BenchmarkQuerierSelect/SortedHead/10000of1000000-4       4020069        4020069        +0.00%
BenchmarkQuerierSelect/SortedHead/100000of1000000-4      4200069        4200069        +0.00%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-4     6000069        6000069        +0.00%
BenchmarkQuerierSelect/Block/1of1000000-4                6000023        6000022        -0.00%
BenchmarkQuerierSelect/Block/10of1000000-4               6000059        6000058        -0.00%
BenchmarkQuerierSelect/Block/100of1000000-4              6000419        6000418        -0.00%
BenchmarkQuerierSelect/Block/1000of1000000-4             6004019        6004018        -0.00%
BenchmarkQuerierSelect/Block/10000of1000000-4            6040019        6040018        -0.00%
BenchmarkQuerierSelect/Block/100000of1000000-4           6400019        6400018        -0.00%
BenchmarkQuerierSelect/Block/1000000of1000000-4          10000020       10000019       -0.00%

benchmark                                                old bytes     new bytes     delta
BenchmarkQuerierSelect/Head/1of1000000-4                 229192200     176001176     -23.21%
BenchmarkQuerierSelect/Head/10of1000000-4                229193352     176002328     -23.21%
BenchmarkQuerierSelect/Head/100of1000000-4               229204872     176013848     -23.21%
BenchmarkQuerierSelect/Head/1000of1000000-4              229320072     176129048     -23.20%
BenchmarkQuerierSelect/Head/10000of1000000-4             230472072     177281048     -23.08%
BenchmarkQuerierSelect/Head/100000of1000000-4            241992072     188801048     -21.98%
BenchmarkQuerierSelect/Head/1000000of1000000-4           357192072     304001048     -14.89%
BenchmarkQuerierSelect/SortedHead/1of1000000-4           229193928     229193928     +0.00%
BenchmarkQuerierSelect/SortedHead/10of1000000-4          229195080     229195080     +0.00%
BenchmarkQuerierSelect/SortedHead/100of1000000-4         229206600     229206600     +0.00%
BenchmarkQuerierSelect/SortedHead/1000of1000000-4        229321800     229321800     +0.00%
BenchmarkQuerierSelect/SortedHead/10000of1000000-4       230473800     230473800     +0.00%
BenchmarkQuerierSelect/SortedHead/100000of1000000-4      241993800     241993800     +0.00%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-4     357193800     357193800     +0.00%
BenchmarkQuerierSelect/Block/1of1000000-4                227201516     227201500     -0.00%
BenchmarkQuerierSelect/Block/10of1000000-4               227202924     227202908     -0.00%
BenchmarkQuerierSelect/Block/100of1000000-4              227217036     227217020     -0.00%
BenchmarkQuerierSelect/Block/1000of1000000-4             227358156     227358140     -0.00%
BenchmarkQuerierSelect/Block/10000of1000000-4            228769356     228769340     -0.00%
BenchmarkQuerierSelect/Block/100000of1000000-4           242881356     242881340     -0.00%
BenchmarkQuerierSelect/Block/1000000of1000000-4          384001616     384001600     -0.00%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-01-28 09:14:56 +00:00
.circleci Add windows tests for query logger (#6653) 2020-01-20 13:17:11 +00:00
.github Add Github Actions workflow to run Prombench (#6703) 2020-01-27 18:06:28 +02:00
cmd Refactor query log fields (#6694) 2020-01-27 09:53:10 +00:00
config Add query log (#6520) 2020-01-08 13:28:43 +00:00
console_libraries Added humanizePercentage formatting to templates (#5670) 2019-06-15 08:59:57 +01:00
consoles Update console template example metric names. (#5678) 2019-06-17 12:13:29 +01:00
discovery Fix order of testutil.Equals (#6695) 2020-01-27 12:21:59 +00:00
docs Added the version to tsdb stats api methods (#6656) 2020-01-19 09:54:39 +00:00
documentation documentation/prometheus-mixin: add dependency on grafonnet 2020-01-11 23:18:04 +01:00
notifier Spelling (#6517) 2020-01-02 15:54:09 +01:00
pkg Add windows tests for query logger (#6653) 2020-01-20 13:17:11 +00:00
prompb Don't sort postings if we only have one block. 2020-01-28 09:14:56 +00:00
promql Don't sort postings if we only have one block. 2020-01-28 09:14:56 +00:00
rules Delete prometheus_rule_group metrics when groups are removed (#6693) 2020-01-27 12:41:32 +00:00
scrape Pass over scrape cache to the next scrape (#6670) 2020-01-22 12:13:47 +00:00
scripts React UI: Serve React UI under /new (#6229) 2019-10-28 10:45:53 +01:00
storage Don't sort postings if we only have one block. 2020-01-28 09:14:56 +00:00
template template: remove redundant parentheses. (#5904) 2019-08-16 09:41:01 +01:00
tsdb Don't sort postings if we only have one block. 2020-01-28 09:14:56 +00:00
util Refactor query log fields (#6694) 2020-01-27 09:53:10 +00:00
vendor web: use chained WithInstrumentation (#6661) 2020-01-19 00:07:10 +00:00
web Don't sort postings if we only have one block. 2020-01-28 09:14:56 +00:00
.dockerignore Added support for s390x image (#6486) 2020-01-13 14:54:07 +01:00
.gitignore Add tsdb/tsdb to .gitignore 2019-11-15 12:41:37 +01:00
.golangci.yml *: enable all default linters (#5504) 2019-05-03 15:11:28 +02:00
.promu.yml Remove Travis as a CI platform (#6369) 2019-11-25 14:59:26 +01:00
CHANGELOG.md Added the version to tsdb stats api methods (#6656) 2020-01-19 09:54:39 +00:00
code-of-conduct.md Add CNCF code of conduct as the Prometheus code of conduct 2016-10-19 21:39:19 +02:00
CONTRIBUTING.md *: bump Go version to 1.13 (#6020) 2019-09-13 13:55:18 +01:00
Dockerfile Integrate beginning of React UI (#5694) 2019-10-17 14:38:09 +02:00
fuzzit.sh Fix fuzzit job (#6005) 2019-09-11 09:08:20 +02:00
go.mod web: use chained WithInstrumentation (#6661) 2020-01-19 00:07:10 +00:00
go.sum web: use chained WithInstrumentation (#6661) 2020-01-19 00:07:10 +00:00
LICENSE Clean up license issues. 2015-01-21 20:07:45 +01:00
MAINTAINERS.md Update and re-work maintainers file. 2019-11-28 19:01:35 +00:00
Makefile Added support for s390x image (#6486) 2020-01-13 14:54:07 +01:00
Makefile.common Bump golangci-lint to v1.18.0 (#6004) 2019-09-10 17:40:08 +02:00
NOTICE Integrate beginning of React UI (#5694) 2019-10-17 14:38:09 +02:00
README.md Update README.md 2020-01-27 18:10:08 +02:00
RELEASE.md Add a note about updating deps 2020-01-20 22:29:12 +01:00
VERSION Cut release 2.15.2 2020-01-06 17:40:15 +00:00

Prometheus

CircleCI Docker Repository on Quay Docker Pulls Go Report Card CII Best Practices fuzzit

Visit prometheus.io for the full documentation, examples and guides.

Prometheus, a Cloud Native Computing Foundation project, is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.

Prometheus's main distinguishing features as compared to other monitoring systems are:

  • a multi-dimensional data model (timeseries defined by metric name and set of key/value dimensions)
  • a flexible query language to leverage this dimensionality
  • no dependency on distributed storage; single server nodes are autonomous
  • timeseries collection happens via a pull model over HTTP
  • pushing timeseries is supported via an intermediary gateway
  • targets are discovered via service discovery or static configuration
  • multiple modes of graphing and dashboarding support
  • support for hierarchical and horizontal federation

Architecture overview

Install

There are various ways of installing Prometheus.

Precompiled binaries

Precompiled binaries for released versions are available in the download section on prometheus.io. Using the latest production release binary is the recommended way of installing Prometheus. See the Installing chapter in the documentation for all the details.

Debian packages are available.

Docker images

Docker images are available on Quay.io or Docker Hub.

You can launch a Prometheus container for trying it out with

$ docker run --name prometheus -d -p 127.0.0.1:9090:9090 prom/prometheus

Prometheus will now be reachable at http://localhost:9090/.

Building from source

To build Prometheus from the source code yourself you need to have a working Go environment with version 1.13 or greater installed. You will also need to have Node.js and Yarn installed in order to build the frontend assets.

You can directly use the go tool to download and install the prometheus and promtool binaries into your GOPATH:

$ go get github.com/prometheus/prometheus/cmd/...
$ prometheus --config.file=your_config.yml

However, when using go get to build Prometheus, Prometheus will expect to be able to read its web assets from local filesystem directories under web/ui/static and web/ui/templates. In order for these assets to be found, you will have to run Prometheus from the root of the cloned repository. Note also that these directories do not include the new experimental React UI unless it has been built explicitly using make assets or make build.

An example of the above configuration file can be found here.

You can also clone the repository yourself and build using make build, which will compile in the web assets so that Prometheus can be run from anywhere:

$ mkdir -p $GOPATH/src/github.com/prometheus
$ cd $GOPATH/src/github.com/prometheus
$ git clone https://github.com/prometheus/prometheus.git
$ cd prometheus
$ make build
$ ./prometheus --config.file=your_config.yml

The Makefile provides several targets:

  • build: build the prometheus and promtool binaries (includes building and compiling in web assets)
  • test: run the tests
  • test-short: run the short tests
  • format: format the source code
  • vet: check the source code for common errors
  • docker: build a docker container for the current HEAD

React UI Development

For more information on building, running, and developing on the new React-based UI, see the React app's README.md.

More information

  • The source code is periodically indexed: Prometheus Core.
  • You will find a CircleCI configuration in .circleci/config.yml.
  • See the Community page for how to reach the Prometheus developers and users on various communication channels.

Contributing

Refer to CONTRIBUTING.md

License

Apache License 2.0, see LICENSE.