Commit graph

13154 commits

Author SHA1 Message Date
Bryan Boreham b6aba4ff14
Merge pull request #14074 from bboreham/kahan-sum-sum
[ENHANCEMENT] PromQL: use Kahan summation for sum()
2024-06-24 11:13:26 +01:00
György Krajcsovits 505ffd34ef Fix lint error
Some weird formatting issue in using comment suggestion

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-06-24 09:33:03 +02:00
George Krajcsovits f45709e710
Update model/histogram/histogram_test.go
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-06-24 07:51:56 +02:00
Martin Chodur 00b110c65c
Fix data corruption in remote write if max_sample_age is applied (#14078)
* fix: try to reproduce the bug from https://github.com/prometheus/prometheus/issues/13979 in a test case

Signed-off-by: David Vavra <sevenood@gmail.com>

* fix: data corruption in remote write if max_sample_age is applied

Signed-off-by: David Vavra <sevenood@gmail.com>

* add benchmark for buildTimeSeries which does the filtering

Signed-off-by: Callum Styan <callumstyan@gmail.com>

---------

Signed-off-by: David Vavra <sevenood@gmail.com>
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: David Vavra <sevenood@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
2024-06-21 14:19:58 -07:00
Bryan Boreham 7a82e4b503 Labels benchmarks: remove artefact of small symbol-tables
Symbol tables with fewer than 128 entries, so everything can be
represented as a single byte, are not realistic.

Stuff the symbol table with fake entries before adding the real ones.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-06-21 16:49:10 +01:00
Bryan Boreham 2ba7bc9446 Labels: further optimisation for dedupelabels
Inline (by copy-paste) the fast path of `decodeVarint` in various
places where it gets called a lot.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-06-21 16:46:13 +01:00
unknown 0d25931049 rebase main and adjust the configuration
Signed-off-by: ouyang1204@gmail.com <ouyang1204@gmail.com>
2024-06-21 19:10:18 +08:00
Bryan Boreham 2ced2f6aec [PERF] Labels: faster varint for dedupelabels
Including tests.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-06-21 11:57:09 +01:00
Piotr d78253319d
queue_manager: add histogram info to error logs (#14326)
Signed-off-by: Piotr Gwizdala <17101802+thampiotr@users.noreply.github.com>
2024-06-20 16:45:13 -07:00
Łukasz Mierzwa dbd29df5df
Fix @goyacc invocation (#14324)
goyacc is installed using 'install-goyacc' and ends up in GOPATH/bin.
GOPATH isn't usually part of standard PATH, so when make tries to run goyacc it fails, unless PATH includes GOPATH/bin.
Other Go tools, like golangci-lint, are also installed via go install into GOPATH/bin but they run correctly because make invocations for them use FIRST_GOPATH viriable to use full path.
Call goyacc using FIRST_GOPATH/bin as well so it works without GOPATH being included in PATH.

Signed-off-by: Lukasz Mierzwa <lukasz@cloudflare.com>
2024-06-20 18:25:44 +01:00
George Krajcsovits a3d92dd72f
Merge pull request #14292 from zenador/nhcb-review-2
[nhcb branch] update missed suggested change from code review
2024-06-20 17:06:28 +02:00
Jeanette Tan dda5f48c9e Merge branch 'main' into nhcb-review-2 2024-06-20 22:50:00 +08:00
Jeanette Tan fc9dc72028 remove eval_with_nhcb
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-20 22:49:00 +08:00
Jeanette Tan a6d788b1be update missed suggested change from code review
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-20 22:49:00 +08:00
akunszt 2aaf99dd0a
discovery: aws: expose Primary IPv6 addresses as label, partially fixes #7406 (#14156)
* discovery: aws: expose Primary IPv6 addresses as label

Add __meta_ec2_primary_ipv6_addresses label. This label contains the
Primary IPv6 address for every ENI attached to the EC2 instance. It is
ordered by the DeviceIndex and the missing elements (interface without
Primary IPv6 address) are kept in the list.

---------

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
2024-06-20 14:36:20 +01:00
George Krajcsovits c25d6d8ac6
Merge pull request #14316 from pracucci/export-labelsToLabelsProto
Export remote.LabelsToLabelsProto() and remote.LabelProtosToLabels()
2024-06-20 08:53:17 +02:00
Marco Pracucci b6dc9c1817
Merge branch 'main' into export-labelsToLabelsProto 2024-06-20 07:35:48 +02:00
machine424 f9ca6c4ae6 chore: add an alert based on the metric prometheus_sd_kubernetes_failures_total
that was introcued in https://github.com/prometheus/prometheus/pull/13554

The same motivation for adding the metric applies: To avoid silent SD failures,
as existing logs may not be regularly checked and can be missed.

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
Co-authored-by: Simon Pasquier <spasquie@redhat.com>
2024-06-19 17:51:56 +02:00
Marco Pracucci 35564c0cb0
Export remote.LabelsToLabelsProto() and remote.LabelProtosToLabels()
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2024-06-19 17:30:49 +02:00
Bryan Boreham 5c417684f8
Merge pull request #14296 from colega/fix-matcher-string-with-empty-label-name
Fix `Matcher.String()` with empty label name
2024-06-19 06:28:53 -04:00
Bryan Boreham 6685cbc234
Merge branch 'main' into fix-matcher-string-with-empty-label-name 2024-06-19 06:13:21 -04:00
Arve Knudsen f45b0fd9ef
Merge pull request #14311 from aknuds1/arve/loggercheck
golangci-lint: Enable loggercheck linter
2024-06-19 12:02:41 +02:00
Bryan Boreham 84602bbace
Merge branch 'main' into fix-matcher-string-with-empty-label-name 2024-06-19 05:56:25 -04:00
Bryan Boreham 5a1886d247
Merge branch 'main' into arve/loggercheck 2024-06-19 05:47:54 -04:00
George Krajcsovits 9a8b6c52ca
Merge pull request #14313 from prometheus/merge-2.53-to-main
Merge 2.53 to main
2024-06-19 10:23:02 +02:00
György Krajcsovits fcabffb999 Merge branch 'release-2.53' into merge-2.53-to-main 2024-06-19 10:06:57 +02:00
machine424 70beda092a fix(notifier): take alertmanagerSet.mtx before checking alertmanagerSet.ams in sendAll
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-06-19 09:43:52 +02:00
machine424 690de487e2 chore(notifier): Split 'Run()' into two goroutines: one to receive target updates and trigger reloads and the other one to send notifications.
This is done to prevent the latter operation from blocking/starving the former, as previously, the `tsets` channel was consumed by the same goroutine that consumes and feeds the buffered `n.more` channel, the `tsets` channel was less likely to be ready as it's unbuffered and only fed every `SDManager.updatert` seconds.

See https://github.com/prometheus/prometheus/issues/13676 and https://github.com/prometheus/prometheus/issues/8768

The synchronization with the sendLoop goroutine is managed through the n.mtx mutex.

This uses a similar approach than scrape manager's efbd6e41c5/scrape/manager.go (L115-L117)

The old TestHangingNotifier was replaced by the new one to more closely reflect reality.

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-06-19 09:43:52 +02:00
machine424 94d28cd6cf chore(notifier): add a reproducer for https://github.com/prometheus/prometheus/issues/13676
to show "targets groups update" starvation when the notifications queue is full and an Alertmanager
is down.

The existing `TestHangingNotifier` that was added in https://github.com/prometheus/prometheus/pull/10948 doesn't really reflect the reality as the SD changes are manually fed into `syncCh` in a continuous way, whereas in reality, updates are only resent every `updatert`.

The test added here sets up an SD manager and links it to the notifier. The SD changes will be triggered by that manager as it's done in reality.

Signed-off-by: machine424 <ayoubmrini424@gmail.com>

Co-authored-by: Ethan Hunter <ehunter@hudson-trading.com>
2024-06-19 09:43:52 +02:00
anarcat 545d31f184
docs: clarify backup requirements for storage (#14297)
* clarify backup requirements for storage

After reading this (again) recently, I was under the impression that our backup strategy ("just throw Bacula at it") was just not good enough and that our backups were inconsistent. I filed [an issue internally][41627] about this because of that concern.

But reading a conversation with @SuperQ on IRC, I came under the impression that only the WAL files would be lost. This is an attempt at documenting this more clearly.

[41627]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/41627
---------

Signed-off-by: anarcat <anarcat@users.noreply.github.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
2024-06-19 07:46:13 +02:00
Arve Knudsen be975bf8d7 golangci-lint: Enable loggercheck linter
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-06-18 20:41:26 +02:00
Björn Rabenstein b6ef745016
Merge pull request #14305 from charleskorn/charleskorn/convert-range-query-tests
promql: Convert more test cases to test scripting language
2024-06-18 17:27:55 +02:00
Björn Rabenstein d968408f51
Merge branch 'main' into charleskorn/convert-range-query-tests 2024-06-18 17:11:57 +02:00
George Krajcsovits d3318c21a3
Merge pull request #14287 from krajorama/nhcb-suggest-fix
native histograms: only reduce resolution for exponential histograms
2024-06-18 16:50:30 +02:00
György Krajcsovits 79020b1e85 Comment float histogram as well
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-06-18 15:22:03 +02:00
György Krajcsovits c309f50ee7 Add comment to state intent of check
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-06-18 15:21:17 +02:00
George Krajcsovits 4c35b9250a
Merge pull request #14303 from prometheus/prepare-2.53.0-release
Prepare 2.53.0 release
2024-06-18 15:08:14 +02:00
Rens Groothuijsen 1c3f322f78
docs: mention implicitly watched directories in documentation (#14019)
* docs: mention implicitly watched directories in documentation

Signed-off-by: Rens Groothuijsen <l.groothuijsen@alumni.maastrichtuniversity.nl>

* Add mention of atomic file renaming

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Rens Groothuijsen <l.groothuijsen@alumni.maastrichtuniversity.nl>

---------

Signed-off-by: Rens Groothuijsen <l.groothuijsen@alumni.maastrichtuniversity.nl>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Co-authored-by: Björn Rabenstein <beorn@grafana.com>
2024-06-18 13:51:47 +02:00
George Krajcsovits 29d3e48267
Update CHANGELOG.md
Co-authored-by: Julien <291750+roidelapluie@users.noreply.github.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-06-18 13:45:53 +02:00
Oleg Zaytsev fd1a89b7c8
Pass affected labels to MemPostings.Delete() (#14307)
* Pass affected labels to MemPostings.Delete

As suggested by @bboreham, we can track the labels of the deleted series
and avoid iterating through all the label/value combinations.

This looks much faster on the MemPostings.Delete call. We don't have a
benchmark on stripeSeries.gc() where we'll pay the price of iterating
the labels of each one of the deleted series.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-06-18 10:28:56 +00:00
Oleg Zaytsev 4f78cc809c
Refactor toNormalisedLower: shorter and slightly faster. (#14299)
Refactor toNormalisedLower: shorter and slightly faster

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-06-18 09:57:37 +00:00
Julien 6572b1fe63
Merge pull request #14306 from pracucci/export-tolabelmatchers
Export remote.ToLabelMatchers()
2024-06-18 11:23:08 +02:00
Marco Pracucci 0fbf4a2529
Export remote.ToLabelMatchers()
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2024-06-17 10:40:45 +02:00
Charles Korn aeec30f082
Convert TestTimestampFunction_StepsMoreOftenThanSamples
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-17 16:56:56 +10:00
Charles Korn 987fa5c6a2
Convert range query test cases to test scripting language
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-17 16:43:01 +10:00
George Krajcsovits 5efc8dd27b
Merge pull request #14302 from yeya24/fix-check-ctx-cancel-count
fix check context cancellation not incrementing count
2024-06-17 08:36:56 +02:00
Ben Ye 0e6fca8e76 add unit test
Signed-off-by: Ben Ye <benye@amazon.com>
2024-06-16 12:09:42 -07:00
György Krajcsovits e121d07388 Prepare release 2.53.0
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-06-16 10:24:09 +02:00
Ben Ye e7db2e30a4 fix check context cancellation not incrementing count
Signed-off-by: Ben Ye <benye@amazon.com>
2024-06-15 11:43:26 -07:00
Oleg Zaytsev 4c1e71fa0b
Reduce the flakiness of TestAsyncRuleEvaluation (#14300)
* Reduce the flakiness of TestAsyncRuleEvaluation

This tests sleeps for 15 millisecond per rule group, and then comprares
the entire execution time to be smaller than a multiple of that delay.

The ruleCount is 6, so it assumes that the test will come to the
assertions in less than 90ms.

Meanwhile, the Github's Windows runner:
- ...Huh, oh? What? How much time? milliwhat? Sorry I don't speak that.

TL;DR, this increases the delay to 250 millisecond. This won't prevent
the test from being flaky, but will reduce the flakiness by several
orders of magnitude and hopefully won't be an issue anymore.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Make tests parallel

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

---------

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-06-14 15:02:46 +02:00