Commit graph

9698 commits

Author SHA1 Message Date
Julien Pivotto 9d63502204 k8s: improve 'own_namespace'
Fail configuration unmarshalling if kubeconfig or api url are set with
"own namespace"

Only read namespace file if needed.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2022-02-01 14:59:09 +01:00
Georg Gadinger c85efa02d9 Fix k8s target discovery when not running inside k8s
When using Kubernetes service discovery on a Prometheus instance that's
not running inside Kubernetes, the creation of the service discovery
fails with a "no such file or directory" error as the special
`/var/run/secrets/kubernetes.io/serviceaccount/namespace` file is not
there.  This commit moves the code that reads this file into the
if-branch where no `APIServer.URL` is given (that one basically makes
Prometheus assume it is running inside of a Kubernetes cluster).

Signed-off-by: Georg Gadinger <nilsding@nilsding.org>
2022-02-01 14:41:25 +01:00
Björn Rabenstein 344a13d960
Merge pull request #10223 from prometheus/beorn7/release
Cut v2.33.0
2022-01-29 22:20:40 +01:00
beorn7 ae8603ce34 Cut v2.33.0
Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-29 16:11:12 +01:00
beorn7 d27e7666cc Bump client_golang to 1.12.1
This pulls in a data-race fix and reduces the excessive number of
histogram buckets in the Go runtime metrics.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-29 16:08:38 +01:00
Björn Rabenstein eb6022fbca
Merge pull request #10183 from prometheus/beorn7/release
Cut v2.33.0-rc.1
2022-01-19 19:26:39 +01:00
beorn7 e00eefa376 Cut v2.33.0-rc.1
Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-19 16:34:49 +01:00
beorn7 ef5815f6f4 Bump github.com/prometheus/client_golang to v1.12.0
Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-19 16:34:49 +01:00
Björn Rabenstein d38bb1cfec
Merge pull request #10182 from prometheus/nexucis/bump-kvsearch
bump @nexucis/kvsearch to v0.4.0
2022-01-19 16:34:02 +01:00
Augustin Husson b9f05e15f7 bump @nexucis/kvsearch to v0.4.0
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2022-01-19 16:11:09 +01:00
Robert Fratto b71a6dbbd1
tsdb/agent: Fix deadlock from simultaneous GC and write (#10166)
* tsdb/agent: Fix deadlock from simultaneous GC and write

This commit fixes a potential deadlock where storing in-memory series
references could deadlock with a WAL GC cycle.

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* add missing license header

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* order local imports

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* align deadlock testing with discovery/manager_test.go method

Also prevents GCs from running concurrently, which could also cause a
deadlock (even though it's currently impossible for two GCs to run
concurrently).

Signed-off-by: Robert Fratto <robertfratto@gmail.com>
2022-01-19 20:23:06 +05:30
Mauro Stettler bf959b36cb
Nits after PR 10051 merge (#10159)
Signed-off-by: Marco Pracucci <marco@pracucci.com>

Co-authored-by: Marco Pracucci <marco@pracucci.com>
2022-01-19 20:20:35 +05:30
Björn Rabenstein 7e74557998
Merge pull request #10157 from prometheus/nexucis/debounce-search
create a component to handle the search bar with debounce
2022-01-19 15:47:41 +01:00
Augustin Husson 74f497ac18 Update web/ui/react-app/src/pages/serviceDiscovery/Services.tsx
Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2022-01-19 14:02:39 +01:00
Björn Rabenstein de51e98128
Merge pull request #10156 from prometheus/beorn7/release
Cut v2.33-rc.0
2022-01-13 15:23:19 +01:00
beorn7 c19e75f725 Cut v2.33-rc.0
Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-13 14:17:57 +01:00
Augustin Husson b9bc8e7c33 create a component to handle the search bar with debounce
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2022-01-12 16:23:38 +01:00
Björn Rabenstein 933f50bcda
Merge pull request #10121 from prometheus/beorn7/promql
PromQL: Promote negative offset and @ modifer to stable
2022-01-12 14:09:46 +01:00
Abhishek 2f4289a3bf
add limit to the rules api (#10152) 2022-01-11 22:44:22 -05:00
Chris Marchbanks eb7c10dbbd
Merge pull request #9274 from bboreham/improve-rw-sharding
remote_write: shard up more when backlogged
2022-01-11 13:23:23 -07:00
beorn7 c2b80d8643 PromQL: Test negative offset in PromQL tests
Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-11 18:23:40 +01:00
beorn7 b39f2739e5 PromQL: Always enable negative offset and @ modifier
This follows the line of argument that the invariant of not looking
ahead of the query time was merely emerging behavior and not a
documented stable feature. Any query that looks ahead of the query
time was simply invalid before the introduction of the negative offset
and the @ modifier.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-11 17:08:55 +01:00
dependabot[bot] 076109fa19
build(deps): bump github.com/aws/aws-sdk-go from 1.42.28 to 1.42.31 (#10146)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.42.28 to 1.42.31.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.42.28...v1.42.31)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-11 15:56:25 +01:00
Ganesh Vernekar 129ed4ec8b
Fix Example() function in TSDB (#10153)
* Fix Example() function in TSDB

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2022-01-11 17:24:03 +05:30
beorn7 61509fc840 PromQL: Promote negative offset and @ modifer to stable
Following the argument that breaking the invariant that PromQL does
not look ahead of the evaluation time implies a breaking change, we
still need to keep the feature flag around, but at least we can
communicate that the feature is considered stable, and that the
feature flags will be ignored from v3 on.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-11 00:34:33 +01:00
dependabot[bot] 714bc3ff29
build(deps): bump google.golang.org/api from 0.63.0 to 0.64.0 (#10135)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.63.0 to 0.64.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.63.0...v0.64.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-10 23:36:26 +01:00
Augustin Husson dd02675467
move dependabot frequency running to monthly (#10144)
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2022-01-10 23:26:24 +01:00
Augustin Husson bff9d06874
Service Discovery Page rework (#10131)
* rework the target page

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* put back the URL of the endpoint

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* replace old code by the new one and change function style

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* align filter and search bar on the same row

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* remove unnecessary return

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* upgrade kvsearch to v0.3.0

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix unit test

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* add missing style on column

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* add placeholder and autofocus

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* put back the previous table design

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix issue relative to the position of the tooltip

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix health filter

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix test on label tooltip

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* simplify filter condition

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* rework service discovery page

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* introduced generic custom infinite scroll component

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* adjust the placeholder in discovery page

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* ignore returning type missing

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* apply fix required by the review

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* index discoveredLabels

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2022-01-10 17:39:51 +01:00
Björn Rabenstein 0f4a1e6eac
Merge pull request #10119 from prometheus/beorn7/remote
API: Promote remote-write-receiver to stable
2022-01-10 15:55:10 +01:00
Augustin Husson dd9f96b893
rework the target page (#10103)
* rework the target page

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* put back the URL of the endpoint

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* replace old code by the new one and change function style

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* align filter and search bar on the same row

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* remove unnecessary return

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* upgrade kvsearch to v0.3.0

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix unit test

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* add missing style on column

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* add placeholder and autofocus

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* put back the previous table design

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix issue relative to the position of the tooltip

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix health filter

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix test on label tooltip

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* simplify filter condition

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2022-01-10 15:53:14 +01:00
Mauro Stettler 0df3489275
Write chunks via queue, predicting the refs (#10051)
* Write chunks via queue, predicting the refs

Our load tests have shown that there is a latency spike in the
remote write handler whenever the head chunks need to be written,
because chunkDiskMapper.WriteChunk() blocks until the chunks are written
to disk.

This adds a queue to the chunk disk mapper which makes the WriteChunk()
method non-blocking unless the queue is full. Reads can still be served
from the queue.

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* address PR feeddback

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* initialize metrics without .Add(0)

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* change isRunningMtx to normal lock

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* do not re-initialize chunkrefmap

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* update metric outside of lock scope

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* add benchmark for adding job to chunk write queue

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* remove unnecessary "success" var

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* gofumpt -extra

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* avoid WithLabelValues call in addJob

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* format comments

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* addressing PR feedback

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* rename cutExpectRef to cutAndExpectRef

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* use head.Init() instead of .initTime()

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* address PR feedback

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* PR feedback

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* update test according to PR feedback

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* replace callbackWg -> awaitCb

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* better test of truncation with empty files

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* replace callbackWg -> awaitCb

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2022-01-10 13:36:45 +00:00
Bryan Boreham 954c0e8020 remote_write: round desired shards up before check
Previously we would reject an increase from 2 to 2.5 as being
within 30%; by rounding up first we see this as an increase from 2 to 3.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-01-10 09:57:37 +00:00
Bryan Boreham 6d01ce8c4d remote_write: shard up more when backlogged
Change the coefficient from 1% to 5%, so instead of targetting to clear
the backlog in 100s we target 20s.

Update unit test to reflect the new behaviour.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-01-10 09:57:37 +00:00
Bryan Boreham d588b14d9c remote_write: detailed test for shard calculation
Drive the input parameters to `calculateDesiredShards()` very precisely,
to illustrate some questionable behaviour marked with `?!`.

See https://github.com/prometheus/prometheus/issues/9178,
https://github.com/prometheus/prometheus/issues/9207,

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-01-10 09:57:37 +00:00
chenlujjj 2ce94ac196
Add '--weight' flag to 'promtool check metrics' command (#10045) 2022-01-07 16:58:28 -05:00
Victor Häggqvist 931acc3ee8
fix overlapping click-targets of alert state checkboxes (#10136)
using the same margin as checkboxes on top of /graph

Signed-off-by: Victor Häggqvist <victor@snilius.com>
2022-01-07 10:38:01 +01:00
Björn Rabenstein 671bb0d5b8
Merge pull request #10128 from prometheus/beorn7/release
Update Go dependencies in preparation for v2.33
2022-01-06 18:48:20 +01:00
Björn Rabenstein 366cba876d
Merge pull request #10110 from prometheus/beorn7/release2
release: Extend instructions for the release shepherd
2022-01-06 18:03:00 +01:00
beorn7 4727c41a4b Update Go dependencies
Note that go.mod has been manually modified to use the following
versions:

- k8s.io/api v0.22.4
- k8s.io/apimachinery v0.22.4
- k8s.io/client-go v0.22.4

The current version of those modules requires Go1.17, which we only
intend to require once Go1.18 is out.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-06 16:22:33 +01:00
Oleg Zaytsev a83d46ee9c
Tidy postingsWithIndexHeap (#10123)
Unexported postingsWithIndexHeap's methods that don't need to be
exported, and added detailed comments.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2022-01-06 16:03:44 +05:30
Bryan Boreham 82860a770c
tsdb: use simpler map key to improve exemplar ingest performance (#10111)
* tsdb: fix exemplar benchmarks

Go benchmarks are expected to do an amount of work that varies with
the `b.N` parameter. Previously these benchmarks would report a result
like 0.01 ns/op, which is nonsense.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* tsdb: use simpler map key to improve exemplar perf

Prometheus holds an index of exemplars so it can discard the oldest one
for a series when a new one is added.
Since the keys are not for human eyes, we can use a simpler format
and save the effort of quoting label values.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Exemplars: allocate index map with estimated size

This avoids Go having to re-size the map several times as it grows.
16 exemplars per series is a guess; if it is too low then the map will
be sparse, while if it is too high then the map will have to resize once
or twice.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-01-06 15:58:58 +05:30
Chris Marchbanks ba03f7fc23
Merge pull request #10102 from prometheus/update-metrics-on-rw-fails
Update sent timestamp when write irrecoverably fails
2022-01-05 10:46:09 -07:00
beorn7 8fdfa52976 API: Promote remote-write-receiver to stable
Since `/api/v1/write` is a mutating endpoint, we should still activate
the remote-write-receiver explicitly. But we should do it in the same
way as the other mutating endpoints, i.e. via a flag
`--web.enable-remote-write-receiver`.

This commit marks the feature flag as deprecated, i.e. it still works
but logs a warning on startup. This enables users to seamlessly
migrate. With the next minor release, we can start ignoring the
feature flag (but still warn a user that is trying to use it).

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-05 15:36:07 +01:00
Shihao Xia d26fd5c97b
fix potential goroutine leaks (#10057)
Signed-off-by: Shihao Xia <charlesxsh@hotmail.com>
2022-01-05 16:21:06 +05:30
Oleg Zaytsev 701545286d
Pop intersected postings heap without popping (#10092)
See this comment for detailed explanation:

https://github.com/prometheus/prometheus/pull/9907#issuecomment-1002189932

TL;DR: if we don't call Pop() on the heap implementation, we don't need
to return our param as an `interface{}` so we save an allocation.
This would be popped for every label value, so it can be thousands of
saved allocations here (see benchmarks).

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2022-01-05 16:16:43 +05:30
beorn7 3be33de316 release: Extend instructions for the release shepherd
This adds the following:
- Find a new volunteer for the release shepherd if needed.
- Consider experimental features for promotion/demotion.

The latter could need more detailed explanation, but I guess we can
add those once we have the planned metrics for feature flags. For now,
I just wanted to have that point in so that it is not completely
forgotten.

Signed-off-by: beorn7 <beorn@grafana.com>

Co-authored-by: Levi Harrison <git@leviharrison.dev>
2022-01-05 11:41:07 +01:00
Björn Rabenstein 5a68a41f13
Merge pull request #10109 from prometheus/beorn7/release
release: Volunteer @beorn7 and @csmarchbanks for 2.33 and 2.34
2022-01-05 11:26:09 +01:00
Peter Štibraný e51a17b501
CompactBlockMetas should produce correct mint/maxt for overlapping blocks. (#10108)
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2022-01-05 15:10:00 +05:30
beorn7 b75843543c release: Volunteer @beorn7 and @csmarchbanks for 2.33 and 2.34
In a meeting at Grafana Labs, we had the shocking revelation that the
next release is scheduled to happen soon, but there is no release
shepherd for it yet.

(Which reminds me: Perhaps we should make it a responsibility of the
release shepherd to make sure there is one for the next release, too?)

Given my current part-time commitment, I would actually try to avoid
being the release shepherd, but on the other hand, it's about time
that a Grafanista to do the job again, and the only one who could do
it at all in the room was me. @csmarchbanks volunteered for the next
release. So here we are.

Of course, if any other team member would like to jump in and do this
release or the next, we'll happily insert you into the list.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-04 16:51:25 +01:00
Goutham Veeramachaneni 6696b7a5f0
Don't update metrics on context cancellation
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2022-01-04 10:46:52 +01:00