The Prometheus monitoring system and time series database.
Find a file
Oleg Zaytsev af0f6da5cb
Fix chunk overflow appending samples at a variable rate (#10607)
* Add a test with variable samples rate append

This test overflows the chunk created in memseries, and the total amount
of samples in the (only) mmapped chunk is 29, instead of the 65565
appended ones.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Cut new chunk when rate prediction was wrong

When appending samples at a slow rate, and then appending at a higher
rate, the prediction we made to cut a new chunk is no longer valid.
Sometimes this can even cause an overflow in the chunk, if more samples
than uint16 can hold are appended.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Improve comment on 2*samplesPerChunk

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Assert that all chunks have less than 240 samples

Also, trigger new chunk at 240, not at more than 240

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2022-04-20 14:54:20 +02:00
.circleci Build the UI when making UI tarball (#10578) 2022-04-12 19:22:51 +01:00
.github Proper support for go modules. (#10486) 2022-04-12 11:00:54 +02:00
cmd Add a --lint flag to the promtool check rules and check config commands (#10435) 2022-04-06 00:05:11 -04:00
config Update Prometheus common (#10492) 2022-03-30 12:49:03 +02:00
console_libraries Make React UI the default, keep old UI under /classic (#8142) 2020-11-03 14:51:48 +01:00
consoles Cleaned up a little bit of HTML 2021-07-28 20:12:06 -04:00
discovery Fix #10507: explicitly include gcp auth from k8s.io to kubernetes discovery 2022-04-01 14:56:37 +03:00
docs Api docs: fix link to time durations 2022-04-11 14:12:21 +02:00
documentation Address code review comments 2022-03-30 09:35:08 +02:00
model Fix OpenMetrics parser to sort uppercase labels correctly. (#10510) 2022-03-31 20:18:53 +02:00
notifier Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
plugins Plugins support (#10495) 2022-03-29 14:44:39 +02:00
prompb Amend Protobuf docstrings referring to model/ packages 2022-04-16 00:37:16 +02:00
promql Changing TotalQueryableSamples from int to int64 (#10549) 2022-04-12 01:22:25 +02:00
rules Add SyncForState Implementation for Ruler HA (#10070) 2022-03-29 02:16:46 +02:00
scrape Merge pull request #10599 from prometheus/release-2.35 2022-04-15 00:10:06 +02:00
scripts Publish UI release artefact in our github releases (#10577) 2022-04-12 19:23:02 +02:00
storage Add SyncForState Implementation for Ruler HA (#10070) 2022-03-29 02:16:46 +02:00
template Build with Go 1.18 (#10501) 2022-03-31 00:16:54 +02:00
tracing Tracing/GRPC: Set TLS credentials only when insecure is false. 2022-04-14 15:18:59 +02:00
tsdb Fix chunk overflow appending samples at a variable rate (#10607) 2022-04-20 14:54:20 +02:00
util Changing TotalQueryableSamples from int to int64 (#10549) 2022-04-12 01:22:25 +02:00
web build(deps): bump @fortawesome/free-solid-svg-icons in /web/ui (#10557) 2022-04-07 15:55:26 +02:00
.dockerignore Add image build for ppc64le architecture 2020-04-06 18:03:58 -03:00
.gitignore Replace vfsgen with embed for static web assets (#10220) 2022-03-09 10:21:31 +01:00
.gitpod.Dockerfile add gitpod dockerfile to upgrade node and npm version 2021-09-27 18:59:41 +02:00
.gitpod.yml fix gitpod by using custome dockerfile and accurate npm ui path 2021-09-27 18:59:41 +02:00
.golangci.yml Enable misspell check in golangci-lint (#10393) 2022-03-03 18:11:19 +01:00
.promu.yml Make the make build depends on npm_licenses task (#7843) 2022-04-08 13:47:59 +02:00
.yamllint Switch golangci-lint to GitHub Actions (#9232) 2021-09-10 16:30:18 +02:00
CHANGELOG.md cut v2.35.0-rc1 (#10596) 2022-04-14 16:41:28 +02:00
CODE_OF_CONDUCT.md Improve repo sync script 2020-06-17 17:12:36 +02:00
CONTRIBUTING.md Change channel to prometheus-dev 2021-09-09 08:13:57 -04:00
Dockerfile Dockerfile: Optimize and consolidate steps (#9180) 2021-09-30 11:13:44 +02:00
go.mod uber.go/automaxprocs package upgrade for cgroups2 support (#10584) 2022-04-13 09:41:28 -04:00
go.sum uber.go/automaxprocs package upgrade for cgroups2 support (#10584) 2022-04-13 09:41:28 -04:00
LICENSE Clean up license issues. 2015-01-21 20:07:45 +01:00
MAINTAINERS.md update email 2021-11-24 08:27:46 -08:00
Makefile Merge pull request #10599 from prometheus/release-2.35 2022-04-15 00:10:06 +02:00
Makefile.common Build with Go 1.18 (#10501) 2022-03-31 00:16:54 +02:00
NOTICE Add license notice for code adapted from Go 2021-12-05 09:01:52 +01:00
plugins.yml Plugins support (#10495) 2022-03-29 14:44:39 +02:00
README.md Add note about Prometheus as a library (#10594) 2022-04-14 22:23:47 +02:00
RELEASE.md Proper support for go modules. (#10486) 2022-04-12 11:00:54 +02:00
SECURITY.md Add SECURITY.md file (#8292) 2021-01-09 09:58:53 +01:00
VERSION cut v2.35.0-rc1 (#10596) 2022-04-14 16:41:28 +02:00

Prometheus

CircleCI Docker Repository on Quay Docker Pulls Go Report Card CII Best Practices Gitpod ready-to-code Fuzzing Status

Visit prometheus.io for the full documentation, examples and guides.

Prometheus, a Cloud Native Computing Foundation project, is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts when specified conditions are observed.

The features that distinguish Prometheus from other metrics and monitoring systems are:

  • A multi-dimensional data model (time series defined by metric name and set of key/value dimensions)
  • PromQL, a powerful and flexible query language to leverage this dimensionality
  • No dependency on distributed storage; single server nodes are autonomous
  • An HTTP pull model for time series collection
  • Pushing time series is supported via an intermediary gateway for batch jobs
  • Targets are discovered via service discovery or static configuration
  • Multiple modes of graphing and dashboarding support
  • Support for hierarchical and horizontal federation

Architecture overview

Install

There are various ways of installing Prometheus.

Precompiled binaries

Precompiled binaries for released versions are available in the download section on prometheus.io. Using the latest production release binary is the recommended way of installing Prometheus. See the Installing chapter in the documentation for all the details.

Docker images

Docker images are available on Quay.io or Docker Hub.

You can launch a Prometheus container for trying it out with

$ docker run --name prometheus -d -p 127.0.0.1:9090:9090 prom/prometheus

Prometheus will now be reachable at http://localhost:9090/.

Building from source

To build Prometheus from source code, You need:

You can directly use the go tool to download and install the prometheus and promtool binaries into your GOPATH:

$ GO111MODULE=on go install github.com/prometheus/prometheus/cmd/...
$ prometheus --config.file=your_config.yml

However, when using go install to build Prometheus, Prometheus will expect to be able to read its web assets from local filesystem directories under web/ui/static and web/ui/templates. In order for these assets to be found, you will have to run Prometheus from the root of the cloned repository. Note also that these directories do not include the new experimental React UI unless it has been built explicitly using make assets or make build.

An example of the above configuration file can be found here.

You can also clone the repository yourself and build using make build, which will compile in the web assets so that Prometheus can be run from anywhere:

$ mkdir -p $GOPATH/src/github.com/prometheus
$ cd $GOPATH/src/github.com/prometheus
$ git clone https://github.com/prometheus/prometheus.git
$ cd prometheus
$ make build
$ ./prometheus --config.file=your_config.yml

The Makefile provides several targets:

  • build: build the prometheus and promtool binaries (includes building and compiling in web assets)
  • test: run the tests
  • test-short: run the short tests
  • format: format the source code
  • vet: check the source code for common errors
  • assets: build the new experimental React UI

Service discovery plugins

Prometheus is bundled with many service discovery plugins. When building Prometheus from source, you can edit the plugins.yml file to disable some service discoveries. The file is a yaml-formated list of go import path that will be built into the Prometheus binary.

After you have changed the file, you need to run make build again.

If you are using another method to compile Prometheus, make plugins will generate the plugins file accordingly.

If you add out-of-tree plugins, which we do not endorse at the moment, additional steps might be needed to adjust the go.mod and go.sum files. As always, be extra careful when loading third party code.

Building the Docker image

The make docker target is designed for use in our CI system. You can build a docker image locally with the following commands:

$ make promu
$ promu crossbuild -p linux/amd64
$ make npm_licenses
$ make common-docker-amd64

NB if you are on a Mac, you will need gnu-tar.

Using Prometheus as a Go Library

Remote Write

We are publishing our Remote Write protobuf independently at buf.build.

You can use that as a library:

$ go get go.buf.build/protocolbuffers/go/prometheus/prometheus

This is experimental.

Prometheus code base

In order to comply with go mod rules, Prometheus release number do not exactly match Go module releases. For the Prometheus v2.y.z releases, we are publishing equivalent v0.y.z tags.

Therefore, a user that would want to use Prometheus v2.35.0 as a library could do:

$ go get github.com/prometheus/prometheus@v0.35.0

This solution makes it clear that we might break our internal Go APIs between minor user-facing releases, as breaking changes are allowed in major version zero.

React UI Development

For more information on building, running, and developing on the new React-based UI, see the React app's README.md.

More information

  • The source code is periodically indexed, but due to an issue with versioning, the "latest" docs shown on Godoc are outdated. Instead, you can use the docs for v2.31.1.
  • You will find a CircleCI configuration in .circleci/config.yml.
  • See the Community page for how to reach the Prometheus developers and users on various communication channels.

Contributing

Refer to CONTRIBUTING.md

License

Apache License 2.0, see LICENSE.