Commit graph

13211 commits

Author SHA1 Message Date
Ben Ye 246b7c6a5c
TSDB: Change block populator to accept postings index function (#14213)
Signed-off-by: Ben Ye <benye@amazon.com>
2024-06-25 09:21:48 +01:00
Ben Ye 5585a3c7e5
tsdb: expose hook to customize block querier (#14114)
* expose hook for block querier

Signed-off-by: Ben Ye <benye@amazon.com>

* update comment

Signed-off-by: Ben Ye <benye@amazon.com>

* use defined type

Signed-off-by: Ben Ye <benye@amazon.com>

---------

Signed-off-by: Ben Ye <benye@amazon.com>
2024-06-25 09:47:06 +02:00
Charles Korn 2c5e88748e
Fix issue where pending OOO read can be left dangling if creating querier fails
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-25 14:22:44 +10:00
Arve Knudsen 0395b04419 golangci-lint: Upgrade to v1.59.1
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-06-24 16:14:22 -07:00
Arve Knudsen d902116b41 Fix various linting errors
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-06-24 16:11:53 -07:00
Bryan Boreham 6030407d25
Merge branch 'main' into elide-queriers
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-06-24 11:17:33 +01:00
Bryan Boreham b6aba4ff14
Merge pull request #14074 from bboreham/kahan-sum-sum
[ENHANCEMENT] PromQL: use Kahan summation for sum()
2024-06-24 11:13:26 +01:00
György Krajcsovits 505ffd34ef Fix lint error
Some weird formatting issue in using comment suggestion

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-06-24 09:33:03 +02:00
George Krajcsovits f45709e710
Update model/histogram/histogram_test.go
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-06-24 07:51:56 +02:00
Martin Chodur 00b110c65c
Fix data corruption in remote write if max_sample_age is applied (#14078)
* fix: try to reproduce the bug from https://github.com/prometheus/prometheus/issues/13979 in a test case

Signed-off-by: David Vavra <sevenood@gmail.com>

* fix: data corruption in remote write if max_sample_age is applied

Signed-off-by: David Vavra <sevenood@gmail.com>

* add benchmark for buildTimeSeries which does the filtering

Signed-off-by: Callum Styan <callumstyan@gmail.com>

---------

Signed-off-by: David Vavra <sevenood@gmail.com>
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: David Vavra <sevenood@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
2024-06-21 14:19:58 -07:00
Bryan Boreham 7a82e4b503 Labels benchmarks: remove artefact of small symbol-tables
Symbol tables with fewer than 128 entries, so everything can be
represented as a single byte, are not realistic.

Stuff the symbol table with fake entries before adding the real ones.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-06-21 16:49:10 +01:00
Bryan Boreham 2ba7bc9446 Labels: further optimisation for dedupelabels
Inline (by copy-paste) the fast path of `decodeVarint` in various
places where it gets called a lot.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-06-21 16:46:13 +01:00
unknown 0d25931049 rebase main and adjust the configuration
Signed-off-by: ouyang1204@gmail.com <ouyang1204@gmail.com>
2024-06-21 19:10:18 +08:00
Bryan Boreham 2ced2f6aec [PERF] Labels: faster varint for dedupelabels
Including tests.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-06-21 11:57:09 +01:00
Piotr d78253319d
queue_manager: add histogram info to error logs (#14326)
Signed-off-by: Piotr Gwizdala <17101802+thampiotr@users.noreply.github.com>
2024-06-20 16:45:13 -07:00
Łukasz Mierzwa dbd29df5df
Fix @goyacc invocation (#14324)
goyacc is installed using 'install-goyacc' and ends up in GOPATH/bin.
GOPATH isn't usually part of standard PATH, so when make tries to run goyacc it fails, unless PATH includes GOPATH/bin.
Other Go tools, like golangci-lint, are also installed via go install into GOPATH/bin but they run correctly because make invocations for them use FIRST_GOPATH viriable to use full path.
Call goyacc using FIRST_GOPATH/bin as well so it works without GOPATH being included in PATH.

Signed-off-by: Lukasz Mierzwa <lukasz@cloudflare.com>
2024-06-20 18:25:44 +01:00
🌲 Harry 🌊 John 🏔 d5f6887294 Pass limit param as hint to storage.Querier
Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>
2024-06-20 09:47:38 -07:00
George Krajcsovits a3d92dd72f
Merge pull request #14292 from zenador/nhcb-review-2
[nhcb branch] update missed suggested change from code review
2024-06-20 17:06:28 +02:00
Jeanette Tan dda5f48c9e Merge branch 'main' into nhcb-review-2 2024-06-20 22:50:00 +08:00
Jeanette Tan fc9dc72028 remove eval_with_nhcb
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-20 22:49:00 +08:00
Jeanette Tan a6d788b1be update missed suggested change from code review
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-20 22:49:00 +08:00
akunszt 2aaf99dd0a
discovery: aws: expose Primary IPv6 addresses as label, partially fixes #7406 (#14156)
* discovery: aws: expose Primary IPv6 addresses as label

Add __meta_ec2_primary_ipv6_addresses label. This label contains the
Primary IPv6 address for every ENI attached to the EC2 instance. It is
ordered by the DeviceIndex and the missing elements (interface without
Primary IPv6 address) are kept in the list.

---------

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
2024-06-20 14:36:20 +01:00
George Krajcsovits c25d6d8ac6
Merge pull request #14316 from pracucci/export-labelsToLabelsProto
Export remote.LabelsToLabelsProto() and remote.LabelProtosToLabels()
2024-06-20 08:53:17 +02:00
Marco Pracucci b6dc9c1817
Merge branch 'main' into export-labelsToLabelsProto 2024-06-20 07:35:48 +02:00
machine424 f9ca6c4ae6 chore: add an alert based on the metric prometheus_sd_kubernetes_failures_total
that was introcued in https://github.com/prometheus/prometheus/pull/13554

The same motivation for adding the metric applies: To avoid silent SD failures,
as existing logs may not be regularly checked and can be missed.

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
Co-authored-by: Simon Pasquier <spasquie@redhat.com>
2024-06-19 17:51:56 +02:00
Marco Pracucci 35564c0cb0
Export remote.LabelsToLabelsProto() and remote.LabelProtosToLabels()
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2024-06-19 17:30:49 +02:00
Bryan Boreham 5c417684f8
Merge pull request #14296 from colega/fix-matcher-string-with-empty-label-name
Fix `Matcher.String()` with empty label name
2024-06-19 06:28:53 -04:00
Bryan Boreham 6685cbc234
Merge branch 'main' into fix-matcher-string-with-empty-label-name 2024-06-19 06:13:21 -04:00
Arve Knudsen f45b0fd9ef
Merge pull request #14311 from aknuds1/arve/loggercheck
golangci-lint: Enable loggercheck linter
2024-06-19 12:02:41 +02:00
Bryan Boreham 84602bbace
Merge branch 'main' into fix-matcher-string-with-empty-label-name 2024-06-19 05:56:25 -04:00
Bryan Boreham 5a1886d247
Merge branch 'main' into arve/loggercheck 2024-06-19 05:47:54 -04:00
George Krajcsovits 9a8b6c52ca
Merge pull request #14313 from prometheus/merge-2.53-to-main
Merge 2.53 to main
2024-06-19 10:23:02 +02:00
György Krajcsovits fcabffb999 Merge branch 'release-2.53' into merge-2.53-to-main 2024-06-19 10:06:57 +02:00
machine424 70beda092a fix(notifier): take alertmanagerSet.mtx before checking alertmanagerSet.ams in sendAll
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-06-19 09:43:52 +02:00
machine424 690de487e2 chore(notifier): Split 'Run()' into two goroutines: one to receive target updates and trigger reloads and the other one to send notifications.
This is done to prevent the latter operation from blocking/starving the former, as previously, the `tsets` channel was consumed by the same goroutine that consumes and feeds the buffered `n.more` channel, the `tsets` channel was less likely to be ready as it's unbuffered and only fed every `SDManager.updatert` seconds.

See https://github.com/prometheus/prometheus/issues/13676 and https://github.com/prometheus/prometheus/issues/8768

The synchronization with the sendLoop goroutine is managed through the n.mtx mutex.

This uses a similar approach than scrape manager's efbd6e41c5/scrape/manager.go (L115-L117)

The old TestHangingNotifier was replaced by the new one to more closely reflect reality.

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-06-19 09:43:52 +02:00
machine424 94d28cd6cf chore(notifier): add a reproducer for https://github.com/prometheus/prometheus/issues/13676
to show "targets groups update" starvation when the notifications queue is full and an Alertmanager
is down.

The existing `TestHangingNotifier` that was added in https://github.com/prometheus/prometheus/pull/10948 doesn't really reflect the reality as the SD changes are manually fed into `syncCh` in a continuous way, whereas in reality, updates are only resent every `updatert`.

The test added here sets up an SD manager and links it to the notifier. The SD changes will be triggered by that manager as it's done in reality.

Signed-off-by: machine424 <ayoubmrini424@gmail.com>

Co-authored-by: Ethan Hunter <ehunter@hudson-trading.com>
2024-06-19 09:43:52 +02:00
anarcat 545d31f184
docs: clarify backup requirements for storage (#14297)
* clarify backup requirements for storage

After reading this (again) recently, I was under the impression that our backup strategy ("just throw Bacula at it") was just not good enough and that our backups were inconsistent. I filed [an issue internally][41627] about this because of that concern.

But reading a conversation with @SuperQ on IRC, I came under the impression that only the WAL files would be lost. This is an attempt at documenting this more clearly.

[41627]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/41627
---------

Signed-off-by: anarcat <anarcat@users.noreply.github.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
2024-06-19 07:46:13 +02:00
Arve Knudsen be975bf8d7 golangci-lint: Enable loggercheck linter
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-06-18 20:41:26 +02:00
Björn Rabenstein b6ef745016
Merge pull request #14305 from charleskorn/charleskorn/convert-range-query-tests
promql: Convert more test cases to test scripting language
2024-06-18 17:27:55 +02:00
Björn Rabenstein d968408f51
Merge branch 'main' into charleskorn/convert-range-query-tests 2024-06-18 17:11:57 +02:00
George Krajcsovits d3318c21a3
Merge pull request #14287 from krajorama/nhcb-suggest-fix
native histograms: only reduce resolution for exponential histograms
2024-06-18 16:50:30 +02:00
György Krajcsovits 79020b1e85 Comment float histogram as well
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-06-18 15:22:03 +02:00
György Krajcsovits c309f50ee7 Add comment to state intent of check
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-06-18 15:21:17 +02:00
George Krajcsovits 4c35b9250a
Merge pull request #14303 from prometheus/prepare-2.53.0-release
Prepare 2.53.0 release
2024-06-18 15:08:14 +02:00
Rens Groothuijsen 1c3f322f78
docs: mention implicitly watched directories in documentation (#14019)
* docs: mention implicitly watched directories in documentation

Signed-off-by: Rens Groothuijsen <l.groothuijsen@alumni.maastrichtuniversity.nl>

* Add mention of atomic file renaming

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Rens Groothuijsen <l.groothuijsen@alumni.maastrichtuniversity.nl>

---------

Signed-off-by: Rens Groothuijsen <l.groothuijsen@alumni.maastrichtuniversity.nl>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Co-authored-by: Björn Rabenstein <beorn@grafana.com>
2024-06-18 13:51:47 +02:00
George Krajcsovits 29d3e48267
Update CHANGELOG.md
Co-authored-by: Julien <291750+roidelapluie@users.noreply.github.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-06-18 13:45:53 +02:00
Oleg Zaytsev fd1a89b7c8
Pass affected labels to MemPostings.Delete() (#14307)
* Pass affected labels to MemPostings.Delete

As suggested by @bboreham, we can track the labels of the deleted series
and avoid iterating through all the label/value combinations.

This looks much faster on the MemPostings.Delete call. We don't have a
benchmark on stripeSeries.gc() where we'll pay the price of iterating
the labels of each one of the deleted series.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-06-18 10:28:56 +00:00
Oleg Zaytsev 4f78cc809c
Refactor toNormalisedLower: shorter and slightly faster. (#14299)
Refactor toNormalisedLower: shorter and slightly faster

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-06-18 09:57:37 +00:00
Julien 6572b1fe63
Merge pull request #14306 from pracucci/export-tolabelmatchers
Export remote.ToLabelMatchers()
2024-06-18 11:23:08 +02:00
Marco Pracucci 0fbf4a2529
Export remote.ToLabelMatchers()
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2024-06-17 10:40:45 +02:00