Commit graph

13279 commits

Author SHA1 Message Date
Charles Korn 7fab72a280
promqltest: add support for setting counter reset hint on histogram samples (#14537)
* promqltest: add support for setting counter reset hint on histogram samples

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-07-31 09:53:05 +02:00
Bartlomiej Plotka 6816149852
Merge pull request #14525 from colega/merge-mmmaxtime-into-shardhash
Store `mmMaxTime` in same field as `seriesShard`
2024-07-31 08:39:38 +02:00
Callum Styan 1561815732
remote write: increase time threshold for resharding (#14450)
Don't reshard if we haven't successfully sent a sample in the last
shardUpdateDuration seconds.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: kushagra Shukla <kushalshukla110@gmail.com>
2024-07-30 14:08:28 -07:00
Max Amin 84b819a69f
feat: add Google cloud roundtripper for remote write (#14346)
* feat: Google Auth for remote write

Signed-off-by: Max Amin <maxamin@google.com>

---------

Signed-off-by: Max Amin <maxamin@google.com>
2024-07-30 16:25:19 +01:00
Bryan Boreham 79a0ba9d64
Merge pull request #13503 from tylitianrui/chore/remove_redundance
remove redundant code
2024-07-30 12:44:03 +01:00
dependabot[bot] 6cef8698c2
build(deps-dev): bump @lezer/generator from 1.7.0 to 1.7.1 in /web/ui (#14382)
Bumps [@lezer/generator](https://github.com/lezer-parser/generator) from 1.7.0 to 1.7.1.
- [Changelog](https://github.com/lezer-parser/generator/blob/main/CHANGELOG.md)
- [Commits](https://github.com/lezer-parser/generator/compare/1.7.0...1.7.1)

---
updated-dependencies:
- dependency-name: "@lezer/generator"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-30 13:30:49 +02:00
Oleg Zaytsev 0300ad58a9
Revert the option regardless of error
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-07-30 11:31:31 +02:00
Oleg Zaytsev d8e1b6bdfd
Store mmMaxTime in same field as seriesShard
We don't use seriesShard during DB initialization, so we can use the
same 8 bytes to store mmMaxTime, and save those during the rest of the
lifetime of the database.

This doesn't affect CPU performance.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-07-30 10:20:29 +02:00
Oleg Zaytsev b7f2f3c3ac
Add BenchmarkLoadRealWLs
This benchmark runs on real WLs rather than fake generated ones.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-07-30 10:19:56 +02:00
George Krajcsovits 395f7088c3
Merge pull request #14514 from fpetkovski/counter-reset-nan
Ignore stale histograms for counter reset detection
2024-07-30 08:43:06 +02:00
Bryan Boreham a7c8ff00c6
Merge pull request #14515 from prometheus/revert-13777-remoteread2
Revert "Chunked remote read: close the querier earlier"
2024-07-29 18:39:19 +01:00
Filip Petkovski 2cd97c61e0
Add more test cases
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2024-07-29 14:53:32 +02:00
Kushal shukla fe12924638
promtool: JUnit-Format XML Test Results (#14506)
* Junit compatible output

Signed-off-by: Kushal Shukla <kushalshukla110@gmail.com>
2024-07-29 21:28:08 +10:00
Marco Pracucci d4f098ae80
Fix relabel.Regexp zero value marshalling (#14517)
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2024-07-26 12:55:39 +00:00
Bryan Boreham 6e89250a5d Revert "Chunked remote read: close the querier earlier"
Believed to trigger segmentation faults due to memory-mapped block
data still being accessed by iterators after the querier is closed.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-26 09:56:38 +01:00
Filip Petkovski be7a4c9b83
Ignore stale histograms for counter reset detection
The histogram stats decoder keeps track of the last seen histogram sample
in order to properly detect counter resets. We are seeing an issue where
a histogram with UnknownResetHint gets treated as a counter reset when it follows
a stale histogram sample.

I believe that this is incorrect since stale samples should be completely ignored
in PromQL. As a result, they should not be stored in the histogram stats iterator
and the counter reset detection needs to be done against the last non-stale sample.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2024-07-26 10:08:31 +02:00
Bryan Boreham 71c90c71d4
Merge pull request #14470 from bboreham/label-sep-byte
Labels: use single byte as separator - small speedup
2024-07-24 13:54:59 +01:00
Bryan Boreham 4dd705d867
Merge pull request #14496 from bboreham/fix-nil-primary
[BUGFIX] Storage: errors from a single secondary querier should be warnings.
2024-07-24 10:15:39 +01:00
Bryan Boreham b03b895a03
Merge pull request #14497 from bboreham/explain-range-queries
[DOCS] Querying basics: explain range and instant queries
2024-07-22 19:50:44 +01:00
Bryan Boreham c037a3df84 lint
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-22 16:34:42 +01:00
Bryan Boreham cc7dcf5afe [DOCS] Querying basics: explain range and instant queries
I often see people ask questions that indicate they don't understand
this point, and launching into "instant vector" and "range vector" is
likely to point them in the wrong direction.

Remove the admonishment that the reader mustn't confuse these things.
Remove mention of "inferred sample timestamps" that is never explained.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-22 16:31:06 +01:00
Bryan Boreham 12bd92a25c [ENHANCEMENT] Storage: Short-circuit merge of single querier with no-op queriers
Filter before checking whether there is only one.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-22 15:35:09 +01:00
Bryan Boreham 90d793e8c5 [BUGFIX] Storage: Single secondary querier errors should be warnings.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-22 15:33:07 +01:00
Bryan Boreham 0ae881739b [TEST] Storage: check MergeQuerier with nil primary
This test fails on current code.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-22 15:02:52 +01:00
Bryan Boreham 677cdcdcec [TEST] Storage: ignore difference between nil and empty
We need this for subsequent changes.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-22 15:01:00 +01:00
Bryan Boreham 23307b02c5 [TESTS] Storage: Improve MergeQuerier tests
`TestMergeQuerierWithSecondaries_ErrorHandling` now tests `NewMergeQuerier`
rather than creating the data structure directly. This means we now test
short-circuiting when only a single querier is required.

Merge `mockGenericQuerier` into `mockQuerier`.
Replace `unwrapMockGenericQuerier` with a visitor pattern.

No change in functionality intended.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-22 14:37:20 +01:00
gotjosh 465891cc56
Rules: Refactor concurrency controller interface (#14491)
* Rules: Refactor concurrency controller interface

Even though the main purpose of this refactor is to modify the interface of the concurrency controller to accept a Context. I did two drive-by modifications that I think are sensible:

1. I have moved the check for dependencies on rules to the controller itself - this aligns with how the controller should behave as it is a deciding factor on wether we should run concurrently or not.
2. I cleaned up some unused methods from the days of the old interface before #13527 changed it.

Signed-off-by: gotjosh <josue.abreu@gmail.com>
---------

Signed-off-by: gotjosh <josue.abreu@gmail.com>
2024-07-22 14:11:18 +01:00
Jan Fajerski 398504e080
Merge pull request #14343 from jan--f/build-status-step
ci: Add job to report build_all status
2024-07-22 09:09:38 +02:00
Goutham Veeramachaneni 1fa9ba838a
Merge pull request #14200 from aknuds1/feat/promote-attributes
OTLP Translator prometheusremotewrite: Support resource attribute promotion
2024-07-21 13:27:23 +02:00
Goutham Veeramachaneni e2ef0dc381
Merge branch 'main' into feat/promote-attributes
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2024-07-21 13:12:17 +02:00
Bartlomiej Plotka a60e5ce362
[PRW 2.0] Added Sender and RW Handler support for Response Stats. (#14444)
* [PRW 2.0] Added Sender support for Response Stats.

Chained on top of https://github.com/prometheus/prometheus/pull/14427
Fixes https://github.com/prometheus/prometheus/issues/14359

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* move write stats to it's own file

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Clean up header usage

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* add missing license to new stats file

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Addressed all comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
2024-07-19 18:53:40 +01:00
Bryan Boreham ac85bd47e1
Merge pull request #14481 from bboreham/update-go-deps
Update Go dependencies
2024-07-19 11:34:28 +01:00
Björn Rabenstein e9dec5fc53
Merge pull request #14483 from prometheus/beorn7/doc
docs: Correct and clarify histogram bucket and resolution limits
2024-07-18 14:31:24 +02:00
Bryan Boreham bdfeae6489
Merge pull request #14474 from bboreham/shrink-memseries
TSDB: shrink memSeries by moving bools together
2024-07-18 12:15:57 +01:00
beorn7 a6fa8a471c docs: Correct and clarify histogram bucket and resolution limits
Previously, the config doc suggested that scrapes are simply failed if
the bucket limit is exceeded. However, instead Prometheus first
attempts to reduce the resolution. This will succeed in almost all
non-pathological cases. However, in the rare case that it doesn't, the
scrape will be failed after all.

For the resolution limit AKA `native_histogram_min_bucket_factor`, the
logic is a bit different. Once the highest factor, i.e. schema -4, is
reached, the scrape will still succeed.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-07-18 12:30:55 +02:00
Arve Knudsen 932918cd3f OTLPConfig.UnmarshalYAML: Return error on invalid input
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-07-18 10:40:47 +02:00
Arve Knudsen ad0a30cdd8 Merge remote-tracking branch 'prometheus/main' into feat/promote-attributes 2024-07-18 10:21:43 +02:00
Bryan Boreham ce3bd4abea Update for Docker deprecation
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-17 17:03:32 +01:00
Bryan Boreham 8f9069f41b Bump Otel semconv version to 1.26.0
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-17 17:00:50 +01:00
Bryan Boreham 6b4ff79ce0 Update Go dependencies
Prior to preparing v2.54 release.

Note not updating to k8s.io 1.30, which would force Go 1.22.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-17 16:30:36 +01:00
Bryan Boreham b75e635374
Merge pull request #14476 from SeanKilleen/patch-2
docs: Update storage.md to provide right-sizing advice on retention
2024-07-17 15:36:28 +01:00
Björn Rabenstein ca7062cf49
Merge pull request #14464 from prometheus/beorn7/histogram
promql: Add NHCB tests
2024-07-17 14:57:46 +02:00
Carrie Edwards a28d1974b4
Merge pull request #14438 from prometheus/cedwards/ooo-chunk-encoding
tsdb: Add support for handling multiple chunks in OOO head
2024-07-16 11:12:23 -07:00
Carrie Edwards 02f05cbf2c Ensure lastMmapRef set to 0 if mmapRefs is empty
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2024-07-16 10:54:06 -07:00
Carrie Edwards 79b53bd3de Refactor TestWBLReplay to use scenarios
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
Co-authored by: Fiona Liao <fiona.liao@grafana.com>:
2024-07-16 10:53:28 -07:00
Carrie Edwards 7e020bb4e9 Refactor
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2024-07-16 10:53:23 -07:00
Carrie Edwards e0d7cef545 Rename some variables in Commit()
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2024-07-16 10:53:23 -07:00
Carrie Edwards fb2ebe1c3d Fix linting
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2024-07-16 10:53:23 -07:00
Carrie Edwards 2e0e4e9ce9 Add support for handling multiple chunks in OOO head
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
Co-authored by: Jeanette Tan <jeanette.tan@grafana.com>:
Co-authored by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>:
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
Co-authored by: Fiona Liao <fiona.liao@grafana.com>:
2024-07-16 10:53:09 -07:00
Arve Knudsen 1d21867d8b Add otlptranslator tests
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-07-16 15:31:52 +02:00