prometheus/.circleci/config.yml
Ganesh Vernekar 095f572d4a
Sync sparsehistogram branch with main (#9189)
* Fix `kuma_sd` targetgroup reporting (#9157)

* Bundle all xDS targets into a single group

Signed-off-by: austin ce <austin.cawley@gmail.com>

* Snapshot in-memory chunks on shutdown for faster restarts (#7229)

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Rename links

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Remove Individual Data Type Caps in Per-shard Buffering for Remote Write (#8921)

* Moved everything to nPending buffer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Simplify exemplar capacity addition

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added pre-allocation

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Don't allocate if not sending exemplars

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Avoid deadlock when processing duplicate series record (#9170)

* Avoid deadlock when processing duplicate series record

`processWALSamples()` needs to be able to send on its output channel
before it can read the input channel, so reads to allow this in case the
output channel is full.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* processWALSamples: update comment

Previous text seems to relate to an earlier implementation.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Optimise WAL loading by removing extra map and caching min-time (#9160)

* BenchmarkLoadWAL: close WAL after use

So that goroutines are stopped and resources released

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* BenchmarkLoadWAL: make series IDs co-prime with #workers

Series are distributed across workers by taking the modulus of the
ID with the number of workers, so multiples of 100 are a poor choice.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* BenchmarkLoadWAL: simulate mmapped chunks

Real Prometheus cuts chunks every 120 samples, then skips those samples
when re-reading the WAL. Simulate this by creating a single mapped chunk
for each series, since the max time is all the reader looks at.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Fix comment

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Remove series map from processWALSamples()

The locks that is commented to reduce contention in are now sharded
32,000 ways, so won't be contended. Removing the map saves memory and
goes just as fast.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* loadWAL: Cache the last mmapped chunk time

So we can skip calling append() for samples it will reject.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Improvements from code review

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Full stops and capitals on comments

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Cache max time in both places mmappedChunks is updated

Including refactor to extract function `setMMappedChunks`, to reduce
code duplication.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Update head min/max time when mmapped chunks added

This ensures we have the correct values if no WAL samples are added for
that series.

Note that `mSeries.maxTime()` was always `math.MinInt64` before, since
that function doesn't consider mmapped chunks.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Split Go and React Tests (#8897)

* Added go-ci and react-ci

Co-authored-by: Julien Pivotto <roidelapluie@inuits.eu>
Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Remove search keymap from new expression editor (#9184)

Signed-off-by: Julius Volz <julius.volz@gmail.com>

Co-authored-by: Austin Cawley-Edwards <austin.cawley@gmail.com>
Co-authored-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Julien Pivotto <roidelapluie@inuits.eu>
Co-authored-by: Bryan Boreham <bjboreham@gmail.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
2021-08-11 15:43:17 +05:30

191 lines
5.3 KiB
YAML

---
version: 2.1
orbs:
prometheus: prometheus/prometheus@0.11.0
go: circleci/go@0.2.0
win: circleci/windows@2.3.0
executors:
# Whenever the Go version is updated here, .promu.yml
# should also be updated.
golang:
docker:
- image: circleci/golang:1.16-node
golang_115:
docker:
- image: circleci/golang:1.15-node
jobs:
test_go:
executor: golang
steps:
- prometheus/setup_environment
- go/load-cache:
key: v1
- run:
command: sudo apt-get install -y yamllint
- run:
command: make GO_ONLY=1
environment:
# Run garbage collection more aggressively to avoid getting OOMed during the lint phase.
GOGC: "20"
# By default Go uses GOMAXPROCS but a Circle CI executor has many
# cores (> 30) while the CPU and RAM resources are throttled. If we
# don't limit this to the number of allocated cores, the job is
# likely to get OOMed and killed.
GOOPTS: "-p 2"
GOMAXPROCS: "2"
GO111MODULE: "on"
- prometheus/check_proto:
version: "3.15.8"
- prometheus/store_artifact:
file: prometheus
- prometheus/store_artifact:
file: promtool
- go/save-cache:
key: v1
- store_test_results:
path: test-results
test_react:
executor: golang
steps:
- checkout
- restore_cache:
keys:
- v3-npm-deps-{{ checksum "web/ui/react-app/yarn.lock" }}
- v3-npm-deps-
- run:
command: make react-app-test
- save_cache:
key: v3-npm-deps-{{ checksum "web/ui/react-app/yarn.lock" }}
paths:
- /home/circleci/.cache/yarn
test_windows:
executor:
name: win/default
shell: powershell
working_directory: /go/src/github.com/prometheus/prometheus
steps:
- checkout
- run:
# Temporary workaround until circleci updates go.
command: |
choco upgrade -y golang
- run:
command: refreshenv
- run:
command: |
$env:GOARCH=""; $env:GOOS=""; cd web/ui; go generate
cd ../..
$TestTargets = go list ./... | Where-Object { $_ -NotMatch "(github.com/prometheus/prometheus/discovery.*|github.com/prometheus/prometheus/config|github.com/prometheus/prometheus/web)"}
go test $TestTargets -vet=off -v
environment:
GOGC: "20"
GOOPTS: "-p 2"
test_tsdb_go115:
executor: golang_115
steps:
- checkout
- run: go test ./tsdb/...
test_mixins:
executor: golang
steps:
- checkout
- run: go install ./cmd/promtool/.
- run:
command: go install -mod=readonly github.com/google/go-jsonnet/cmd/jsonnet github.com/google/go-jsonnet/cmd/jsonnetfmt github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb
working_directory: ~/project/documentation/prometheus-mixin
- run:
command: make clean
working_directory: ~/project/documentation/prometheus-mixin
- run:
command: jb install
working_directory: ~/project/documentation/prometheus-mixin
- run:
command: sudo apt-get install -y yamllint
- run:
command: make
working_directory: ~/project/documentation/prometheus-mixin
- run:
command: git diff --exit-code
working_directory: ~/project/documentation/prometheus-mixin
repo_sync:
executor: golang
steps:
- checkout
- run: mkdir -v -p "${PATH%%:*}" && curl -sL --fail https://github.com/mikefarah/yq/releases/download/v4.6.3/yq_linux_amd64 -o "${PATH%%:*}/yq" && chmod -v +x "${PATH%%:*}/yq"
- run: sha256sum -c <(echo "c4343783c3361495c0d6d1eb742bba7432aa65e13e9fb8d7e201d544bcf14246 ${PATH%%:*}/yq")
- run: ./scripts/sync_repo_files.sh
workflows:
version: 2
prometheus:
jobs:
- test_go:
filters:
tags:
only: /.*/
- test_react:
filters:
tags:
only: /.*/
- test_tsdb_go115:
filters:
tags:
only: /.*/
- test_mixins:
filters:
tags:
only: /.*/
- test_windows:
filters:
tags:
only: /.*/
- prometheus/build:
name: build
parallelism: 12
filters:
tags:
only: /^v[0-9]+(\.[0-9]+){2}(-.+|[^-.]*)$/
- prometheus/publish_main:
context: org-context
requires:
- test_go
- test_react
- build
filters:
branches:
only: main
image: circleci/golang:1-node
- prometheus/publish_release:
context: org-context
requires:
- test_go
- test_react
- build
filters:
tags:
only: /^v[0-9]+(\.[0-9]+){2}(-.+|[^-.]*)$/
branches:
ignore: /.*/
image: circleci/golang:1-node
nightly:
triggers:
- schedule:
cron: "0 0 * * *"
filters:
branches:
only:
- main
jobs:
- repo_sync:
context: org-context