Commit graph

9798 commits

Author SHA1 Message Date
Marco Pracucci 583e746a82
Fix error reported by asyncBlockWriter.addSeries()
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2022-02-10 15:07:13 +01:00
Peter Štibraný cd86e92b74
Merge pull request #132 from grafana/reintroduce-old-chunk-mapper
Partial revert of PR 109 – This reintroduces old chunk disk mapper without a queue, that is used when queue size is configured to 0.
2022-02-10 14:33:35 +01:00
Dimitar Dimitrov 644605deb5
Merge pull request #113 from grafana/dimitar/compare-group-src-tenants-w-dups
Account for repeating tenants when comparing rules
2022-01-24 13:49:58 +01:00
Dimitar Dimitrov f05e435f6a
Remove copying of locked structs in test 2022-01-21 15:12:45 +01:00
Dimitar Dimitrov eae4ac40ba
Add tests for rules.Group.Equals 2022-01-21 15:07:20 +01:00
Dimitar Dimitrov 16faee8b78
Account for repeating tenants when comparing rules 2022-01-21 14:21:50 +01:00
Mauro Stettler bc7a802871
Merge pull request #109 from grafana/update_upstream
Merge upstream main into private fork
2022-01-12 17:44:28 -03:00
Mauro Stettler ff2443c6b9
addressing PR feedback
Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>
2022-01-12 20:06:32 +00:00
Mauro Stettler 33a0fb58d5
fixing merge mistakes
Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>
2022-01-12 19:21:46 +00:00
Mauro Stettler 6ed72dadca
fixing merge mistakes
Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>
2022-01-12 18:49:24 +00:00
Mauro Stettler f4d628d419
resolving merge conflicts 2022-01-10 19:41:06 +00:00
Björn Rabenstein 0f4a1e6eac
Merge pull request #10119 from prometheus/beorn7/remote
API: Promote remote-write-receiver to stable
2022-01-10 15:55:10 +01:00
Augustin Husson dd9f96b893
rework the target page (#10103)
* rework the target page

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* put back the URL of the endpoint

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* replace old code by the new one and change function style

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* align filter and search bar on the same row

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* remove unnecessary return

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* upgrade kvsearch to v0.3.0

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix unit test

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* add missing style on column

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* add placeholder and autofocus

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* put back the previous table design

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix issue relative to the position of the tooltip

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix health filter

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix test on label tooltip

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* simplify filter condition

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2022-01-10 15:53:14 +01:00
Mauro Stettler 0df3489275
Write chunks via queue, predicting the refs (#10051)
* Write chunks via queue, predicting the refs

Our load tests have shown that there is a latency spike in the
remote write handler whenever the head chunks need to be written,
because chunkDiskMapper.WriteChunk() blocks until the chunks are written
to disk.

This adds a queue to the chunk disk mapper which makes the WriteChunk()
method non-blocking unless the queue is full. Reads can still be served
from the queue.

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* address PR feeddback

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* initialize metrics without .Add(0)

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* change isRunningMtx to normal lock

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* do not re-initialize chunkrefmap

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* update metric outside of lock scope

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* add benchmark for adding job to chunk write queue

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* remove unnecessary "success" var

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* gofumpt -extra

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* avoid WithLabelValues call in addJob

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* format comments

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* addressing PR feedback

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* rename cutExpectRef to cutAndExpectRef

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* use head.Init() instead of .initTime()

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* address PR feedback

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* PR feedback

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* update test according to PR feedback

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* replace callbackWg -> awaitCb

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* better test of truncation with empty files

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

* replace callbackWg -> awaitCb

Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2022-01-10 13:36:45 +00:00
chenlujjj 2ce94ac196
Add '--weight' flag to 'promtool check metrics' command (#10045) 2022-01-07 16:58:28 -05:00
Peter Štibraný 049c5e4435
Merge pull request #104 from grafana/56quarters/upstream-update
Upgrade prometheus to latest main
2022-01-07 16:38:01 +01:00
Nick Pillitteri fc643a4854 Merge remote-tracking branch 'upstream/main' into 56quarters/upstream-update 2022-01-07 10:18:20 -05:00
Victor Häggqvist 931acc3ee8
fix overlapping click-targets of alert state checkboxes (#10136)
using the same margin as checkboxes on top of /graph

Signed-off-by: Victor Häggqvist <victor@snilius.com>
2022-01-07 10:38:01 +01:00
Björn Rabenstein 671bb0d5b8
Merge pull request #10128 from prometheus/beorn7/release
Update Go dependencies in preparation for v2.33
2022-01-06 18:48:20 +01:00
Björn Rabenstein 366cba876d
Merge pull request #10110 from prometheus/beorn7/release2
release: Extend instructions for the release shepherd
2022-01-06 18:03:00 +01:00
beorn7 4727c41a4b Update Go dependencies
Note that go.mod has been manually modified to use the following
versions:

- k8s.io/api v0.22.4
- k8s.io/apimachinery v0.22.4
- k8s.io/client-go v0.22.4

The current version of those modules requires Go1.17, which we only
intend to require once Go1.18 is out.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-06 16:22:33 +01:00
Oleg Zaytsev a83d46ee9c
Tidy postingsWithIndexHeap (#10123)
Unexported postingsWithIndexHeap's methods that don't need to be
exported, and added detailed comments.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2022-01-06 16:03:44 +05:30
Bryan Boreham 82860a770c
tsdb: use simpler map key to improve exemplar ingest performance (#10111)
* tsdb: fix exemplar benchmarks

Go benchmarks are expected to do an amount of work that varies with
the `b.N` parameter. Previously these benchmarks would report a result
like 0.01 ns/op, which is nonsense.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* tsdb: use simpler map key to improve exemplar perf

Prometheus holds an index of exemplars so it can discard the oldest one
for a series when a new one is added.
Since the keys are not for human eyes, we can use a simpler format
and save the effort of quoting label values.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Exemplars: allocate index map with estimated size

This avoids Go having to re-size the map several times as it grows.
16 exemplars per series is a guess; if it is too low then the map will
be sparse, while if it is too high then the map will have to resize once
or twice.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-01-06 15:58:58 +05:30
Chris Marchbanks ba03f7fc23
Merge pull request #10102 from prometheus/update-metrics-on-rw-fails
Update sent timestamp when write irrecoverably fails
2022-01-05 10:46:09 -07:00
Peter Štibraný 19b1932023
Merge pull request #103 from grafana/upgrade-prometheus
Upgrade prometheus to latest main
2022-01-05 17:35:03 +01:00
beorn7 8fdfa52976 API: Promote remote-write-receiver to stable
Since `/api/v1/write` is a mutating endpoint, we should still activate
the remote-write-receiver explicitly. But we should do it in the same
way as the other mutating endpoints, i.e. via a flag
`--web.enable-remote-write-receiver`.

This commit marks the feature flag as deprecated, i.e. it still works
but logs a warning on startup. This enables users to seamlessly
migrate. With the next minor release, we can start ignoring the
feature flag (but still warn a user that is trying to use it).

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-05 15:36:07 +01:00
Shihao Xia d26fd5c97b
fix potential goroutine leaks (#10057)
Signed-off-by: Shihao Xia <charlesxsh@hotmail.com>
2022-01-05 16:21:06 +05:30
Oleg Zaytsev 701545286d
Pop intersected postings heap without popping (#10092)
See this comment for detailed explanation:

https://github.com/prometheus/prometheus/pull/9907#issuecomment-1002189932

TL;DR: if we don't call Pop() on the heap implementation, we don't need
to return our param as an `interface{}` so we save an allocation.
This would be popped for every label value, so it can be thousands of
saved allocations here (see benchmarks).

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2022-01-05 16:16:43 +05:30
beorn7 3be33de316 release: Extend instructions for the release shepherd
This adds the following:
- Find a new volunteer for the release shepherd if needed.
- Consider experimental features for promotion/demotion.

The latter could need more detailed explanation, but I guess we can
add those once we have the planned metrics for feature flags. For now,
I just wanted to have that point in so that it is not completely
forgotten.

Signed-off-by: beorn7 <beorn@grafana.com>

Co-authored-by: Levi Harrison <git@leviharrison.dev>
2022-01-05 11:41:07 +01:00
Björn Rabenstein 5a68a41f13
Merge pull request #10109 from prometheus/beorn7/release
release: Volunteer @beorn7 and @csmarchbanks for 2.33 and 2.34
2022-01-05 11:26:09 +01:00
Peter Štibraný 61e6173900 Merge remote-tracking branch 'upstream/main' into upgrade-prometheus 2022-01-05 10:44:37 +01:00
Peter Štibraný e51a17b501
CompactBlockMetas should produce correct mint/maxt for overlapping blocks. (#10108)
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2022-01-05 15:10:00 +05:30
Peter Štibraný 3117005b22
Extract interface from ActivityQueryTracker and allows passing custom implementation (#102)
* Extract QueryTracker interface.

* Update from Prometheus PR 10071.

* Update promql/engine.go

Co-authored-by: Mauro Stettler <mauro.stettler@gmail.com>

* Update promql/engine.go

Co-authored-by: Mauro Stettler <mauro.stettler@gmail.com>

Co-authored-by: Mauro Stettler <mauro.stettler@gmail.com>
2022-01-05 10:34:15 +01:00
beorn7 b75843543c release: Volunteer @beorn7 and @csmarchbanks for 2.33 and 2.34
In a meeting at Grafana Labs, we had the shocking revelation that the
next release is scheduled to happen soon, but there is no release
shepherd for it yet.

(Which reminds me: Perhaps we should make it a responsibility of the
release shepherd to make sure there is one for the next release, too?)

Given my current part-time commitment, I would actually try to avoid
being the release shepherd, but on the other hand, it's about time
that a Grafanista to do the job again, and the only one who could do
it at all in the room was me. @csmarchbanks volunteered for the next
release. So here we are.

Of course, if any other team member would like to jump in and do this
release or the next, we'll happily insert you into the list.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-04 16:51:25 +01:00
Goutham Veeramachaneni 6696b7a5f0
Don't update metrics on context cancellation
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2022-01-04 10:46:52 +01:00
Chris Marchbanks dfa5cb7462
Merge pull request #10038 from charlesxsh/fix-TestReshardRaceWithStop
add proper exit for loop
2022-01-03 09:02:45 -07:00
Peter Štibraný 5205ff1a81
Merge pull request #95 from grafana/update-prometheus-2022-01-03
Update prometheus to `a961062c37f1675d5925d8b74ddda0a6322ed82d`
2022-01-03 12:47:14 +01:00
Oleg Zaytsev 3f510ea6c9 Merge prometheus/a961062 2022-01-03 12:00:39 +01:00
Goutham Veeramachaneni 1af81dc5c9
Update sent timestamp when write irrecoverably fails.
We have an alert that fires when prometheus_remote_storage_highest_timestamp_in_seconds - prometheus_remote_storage_queue_highest_sent_timestamp_seconds
becomes too high. But we have an agent that fires this when the remote "rate-limits" the user.

This is because prometheus_remote_storage_queue_highest_sent_timestamp_seconds doesn't get updated
when the remote sends a 429.

I think we should update the metrics, and the change I made makes sense. Because if the requests fails
because of connectivity issues, etc. we will never exit the `sendWriteRequestWithBackoff` function. It only
exits the function when there is a non-recoverable error, like a bad status code, and in that case, I think
the metric needs to be updated.

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2022-01-03 11:13:48 +01:00
David Leadbeater a961062c37
Disable time based retention in tests (#8818)
Fixes #7699.

Signed-off-by: David Leadbeater <dgl@dgl.cx>
2022-01-02 23:46:03 +01:00
Drew DeVault 8b0e1efd4f
Merge pull request #10100 from ddevault/main
codemirror-promql: use npm to run lezer-generator
2022-01-02 11:41:47 +01:00
Ben Kochie b277571b24
Merge pull request #10097 from prometheus/superq/bump_go
Update minimum Go version
2022-01-02 08:56:55 +01:00
Jessica G 174a1147d5
Merge pull request #9861 from JessicaGreben/minor-prom-improvements
Add exit code constants in promtool
2021-12-31 12:07:02 -08:00
SuperQ a78a3cea02
Rollback k8s.io to 0.22.4.
Signed-off-by: SuperQ <superq@gmail.com>
2021-12-31 17:39:40 +01:00
SuperQ 41b43c3b87
Update minimum Go version
Update minimum required Go version to 1.16 due to use of standard
library package `io/fs`.

Signed-off-by: SuperQ <superq@gmail.com>
2021-12-31 17:29:00 +01:00
jessicagreben 4b03fa3100 replace config exit code with failure exit code
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-12-30 05:37:57 -08:00
Shihao Xia c3e7bfb813 add proper exit for loop
Signed-off-by: Shihao Xia <charlesxsh@hotmail.com>
2021-12-29 23:48:11 -05:00
dependabot[bot] 51b0826925
build(deps): bump github.com/aws/aws-sdk-go from 1.42.24 to 1.42.25 (#10093)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-28 16:56:50 -05:00
dependabot[bot] e0cec62025
build(deps): bump github.com/Azure/azure-sdk-for-go from 60.2.0+incompatible to 60.3.0+incompatible (#10086)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-28 12:46:04 -05:00
dependabot[bot] 440974f403
build(deps): bump github.com/Azure/go-autorest/autorest/adal from 0.9.17 to 0.9.18 (#10089)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-28 11:56:21 -05:00