Commit graph

12087 commits

Author SHA1 Message Date
dependabot[bot] ec6253d283
build(deps): bump github.com/alecthomas/kingpin/v2
Bumps [github.com/alecthomas/kingpin/v2](https://github.com/alecthomas/kingpin) from 2.3.2 to 2.4.0.
- [Release notes](https://github.com/alecthomas/kingpin/releases)
- [Commits](https://github.com/alecthomas/kingpin/compare/v2.3.2...v2.4.0)

---
updated-dependencies:
- dependency-name: github.com/alecthomas/kingpin/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 23:28:01 +00:00
dependabot[bot] 59f467ca84
build(deps): bump bufbuild/buf-setup-action from 1.26.1 to 1.28.1
Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.26.1 to 1.28.1.
- [Release notes](https://github.com/bufbuild/buf-setup-action/releases)
- [Commits](eb60cd0de4...382440cdb8)

---
updated-dependencies:
- dependency-name: bufbuild/buf-setup-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 23:24:02 +00:00
dependabot[bot] 6d8e82df1e
build(deps): bump github/codeql-action from 2.22.5 to 2.22.8
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.22.5 to 2.22.8.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](74483a38d3...407ffafae6)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 23:23:56 +00:00
dependabot[bot] 2f5f711cf1
build(deps): bump dessant/lock-threads from 4.0.1 to 5.0.1
Bumps [dessant/lock-threads](https://github.com/dessant/lock-threads) from 4.0.1 to 5.0.1.
- [Release notes](https://github.com/dessant/lock-threads/releases)
- [Changelog](https://github.com/dessant/lock-threads/blob/main/CHANGELOG.md)
- [Commits](be8aa5be94...1bf7ec2505)

---
updated-dependencies:
- dependency-name: dessant/lock-threads
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 23:23:47 +00:00
Julien Levesy e4ec263bcc
fix(wlog/watcher): read segment synchronously when not tailing (#13224)
Signed-off-by: Julien Levesy <jlevesy@gmail.com>
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
2023-12-01 14:26:38 -08:00
Julien Levesy 501f514389
feat(tsdb/agent): notify remote storage when commit happens (#13223)
Signed-off-by: Julien Levesy <jlevesy@gmail.com>
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
2023-12-01 14:00:26 -08:00
Julien Pivotto 52b15d2195
Merge pull request #13219 from djcode/main
Discovery: Add support for NS records to DNS Discovery
2023-12-01 16:10:07 +01:00
Julien Pivotto 6e8e16c050
Merge pull request #13227 from prometheus/release-2.48
Merge release-2.48 back into main
2023-12-01 16:09:02 +01:00
Jeanette Tan 2910b48180 Make mad_over_time experimental and move tests
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-12-01 01:56:07 +08:00
Jeanette Tan 9bf4cc993e Add mad_over_time function
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-12-01 01:22:58 +08:00
Björn Rabenstein 5dbbadf598
Merge pull request #13216 from prometheus/beorn7/doc
Update “conventional histogram” → “classic histogram”
2023-11-30 10:35:27 +01:00
Daniel Jolly 9ebea5e1ca Removed extra whitespace
Signed-off-by: Daniel Jolly <code@danieljolly.com>
2023-11-29 12:50:09 -05:00
Oleksandr Redko 2a75604f8e
Enable default revive rules (#13068)
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-11-29 17:23:34 +00:00
Daniel Jolly 2c420b518e Add support for NS records in DNS discovery
Signed-off-by: Daniel Jolly <code@danieljolly.com>
2023-11-29 12:21:50 -05:00
beorn7 0eb0ca42c5 Update “conventional histogram” → “classic histogram”
Signed-off-by: beorn7 <beorn@grafana.com>
2023-11-29 15:22:58 +01:00
Björn Rabenstein f216ddadbc
Merge pull request #13210 from prometheus/bwplotka-patch-1
Added Bartek as the next release shepherd.
2023-11-29 14:49:38 +01:00
Fiona Liao b8bcaef14d
Fix histogram append errors (#13201)
* Fix histogram append errors

We should check counterReset condition rather than okToAppend because if
there's a counter reset, okToAppend is always set to false.

Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com>
2023-11-29 11:39:12 +01:00
Fiona Liao ce126230e7
Fix chunks iterator bug when tombstone covers a whole chunk (#13209)
When no samples are returned in a chunk because all the samples have
been deleted, the chunk iterator then stops without iterating through
any remaining chunks.

Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com>
2023-11-29 11:24:04 +01:00
Linas Medziunas 048886ae8a Histograms: optimize floatBucketIterator for common case
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-29 08:54:05 +02:00
Bartlomiej Plotka 911d0d0f7d
Update RELEASE.md
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2023-11-28 20:19:12 +00:00
Bartlomiej Plotka 34812fa759
Added Bartek as the next release shepherd.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2023-11-28 20:18:19 +00:00
Björn Rabenstein 97f9ad9d31
Merge pull request #13191 from linasm/reduceResolution-inplace
Reuse receiver slices in [Float]Histogram.ReduceResolution
2023-11-28 17:48:52 +01:00
Björn Rabenstein 980e2895a2
Merge pull request #13129 from fatsheep9146/reduce-resolution-automatically
Native Histograms: automatically reduce resolution rather than fail scrape
2023-11-28 17:26:36 +01:00
Julien Pivotto 4293de96c7
Merge pull request #11299 from galexrt/sort_by_labels
promql: add sort_by_label and sort_by_label_desc functions
2023-11-28 15:31:48 +01:00
Julien Pivotto c1ec6ae851 sort_by_label: Switch to feature flag
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-11-28 15:10:12 +01:00
Xiaochao Dong 28d8f1650c
tsdb: Make sure the cache for postings cardinality properly honors the label name (#12653)
Add a string remembering which label and limit the cache corresponds to.

Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>
2023-11-28 13:54:37 +00:00
Linas Medziunas 49613823f8 Reuse slices in [Float]Histogram.ReduceResolution
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-28 15:45:31 +02:00
Arve Knudsen 1200c89d0c
Fix tsdb.stripeSeries.gc so it handles conflicts properly (#13195)
* Fix tsdb.stripeSeries.gc so it handles conflicts properly

tsdb.stripeSeries.gc needs to prune seriesHashmap.conflicts first,
otherwise seriesHashmap replaces the unique field with the first among
the conflicts. Also add regression test.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* TestStripeSeries_gc: Support stringlabels, don't use internals

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-28 14:43:35 +01:00
Alexander Trost 5051a993ab promql: add sort_by_label and sort_by_label_desc functions
This adds functions to sort a vector by its label value.

Based on https://github.com/prometheus/prometheus/pull/1533

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2023-11-28 14:40:07 +01:00
Björn Rabenstein a6d4b8d97b
Merge pull request #13190 from linasm/fix-float-histogram-add-sub-mutating-argument
Fix FloatHistogram.Add/Sub mutating its argument
2023-11-28 14:04:50 +01:00
Björn Rabenstein 5f6c060284
Merge pull request #13189 from linasm/explicit-schema-check-in-histogram-reduce-resolution
Explicit schema check in [Float]Histogram.ReduceResolution
2023-11-28 14:00:34 +01:00
dependabot[bot] 8d058884a0
build(deps): bump sanitize-html from 2.10.0 to 2.11.0 in /web/ui (#12513)
Bumps [sanitize-html](https://github.com/apostrophecms/sanitize-html) from 2.10.0 to 2.11.0.
- [Changelog](https://github.com/apostrophecms/sanitize-html/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apostrophecms/sanitize-html/compare/2.10.0...2.11.0)

---
updated-dependencies:
- dependency-name: sanitize-html
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-28 12:49:34 +01:00
David Peleg 969a9b7090
Fix docs: in labelname, convert to underscore (#12555)
Fixes https://github.com/prometheus/docs/issues/735

Signed-off-by: David Peleg <107049994+dapeleg-dn@users.noreply.github.com>
2023-11-28 12:30:05 +01:00
Bryan Boreham 1b92fb10de
Improvements to PR template (#13198)
I like a bit more of a hint what the PR is supposed to achieve.

Also, I took out a suggestion not to write tests.  We can decide
case-by-case if it's ok not to add a test; no need to encourage it.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-28 10:52:27 +00:00
Fiona Liao 5bee0cfce2
Change ChunkReader.Chunk() to ChunkOrIterable()
The ChunkReader interface's Chunk() has been changed to ChunkOrIterable(). 

This is a precursor to OOO native histogram support - with OOO native histograms, the chunks.Meta passed to Chunk() can result in multiple chunks being returned rather than just a single chunk (e.g. if oooMergedChunk has a counter reset in the middle). 

To support this, ChunkOrIterable() requires either a single chunk or an iterable to be returned. If an iterable is returned, the caller has the responsibility of converting the samples from the iterable into possibly multiple chunks. The OOOHeadChunkReader now returns an iterable rather than a chunk to prepare for the native histograms case. Also as a beneficial side effect, oooMergedChunk and boundedChunk has been simplified as they only need to implement the Iterable interface now, not the full Chunk interface.

---------

Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2023-11-28 11:14:29 +01:00
Arve Knudsen ecc37588b0
tsdb: seriesHashmap.set by making receiver a pointer (#13193)
* Fix tsdb.seriesHashmap.set by making receiver a pointer

The method tsdb.seriesHashmap.set currently doesn't set the conflicts
field properly, due to the receiver being a non-pointer. Fix by turning
the receiver into a pointer, and add a corresponding regression test.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-27 15:40:30 +00:00
Ziqi Zhao 19ecc5dd94 add test case for bigGap
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-11-26 22:20:44 +08:00
Linas Medziunas f99ecc376e Fix FloatHistogram.Add/Sub mutating its argument
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-26 09:26:34 +02:00
Linas Medziunas ea1862aab4 Explicit schema check in [Float]Histogram.ReduceResolution
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-25 11:38:15 +02:00
Julien Pivotto 965e603fa7
Merge pull request #13184 from bboreham/exemplar-sort
Scraping: use slices.sort for exemplars
2023-11-25 09:34:48 +01:00
zenador ccfe14d7e7
PromQL: ignore small errors for bucketQuantile (#13153)
promql: Improve histogram_quantile calculation for classic buckets

Tiny differences between classic buckets are most likely caused by floating point precision issues. With this commit, relative changes below a certain threshold are ignored. This makes the result of histogram_quantile more meaningful, and also avoids triggering the _input to histogram_quantile needed to be fixed for monotonicity_ annotations in unactionable cases.

This commit also adds explanation of the new adjustment and of the monotonicity annotation to the documentation of `histogram_quantile`.

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-25 00:05:38 +01:00
Yury Molodov 2e205ee95c
ui: heatmap visualization for histogram buckets (#13096)
ui: heatmap visualization for histogram buckets

Signed-off-by: Yury Moladau <yurymolodov@gmail.com>

---------

Signed-off-by: Yury Moladau <yurymolodov@gmail.com>
2023-11-24 22:44:48 +01:00
Julien Pivotto eda73dd3e5
Merge pull request #13187 from bboreham/refactor-newscrapeloop
Scraping tests: refactor scrapeLoop creation
2023-11-24 19:48:44 +01:00
Bryan Boreham 3e287e0170 Scraping tests: refactor scrapeLoop creation
Pull boilerplate code into a function. Where appropriate we set some
config on the returned object.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-24 17:28:09 +00:00
Bryan Boreham 784a2d2c74
Merge pull request #12992 from bboreham/single-scrape-buffer-pool
Scraping: share buffer pool across all scrapes
2023-11-24 16:26:19 +00:00
Bryan Boreham f0e1b592ab Scraping: use slices.sort for exemplars
The sort implementation using Go generics is used everywhere else
in Prometheus.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-24 14:42:26 +00:00
Charles Korn 59844498f7
Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115)
* Add failing test.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Don't run OOO head garbage collection while reads are running.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Add further test cases for different order of operations.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Ensure all queriers are closed if `DB.blockChunkQuerierForRange()` fails.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Ensure all queriers are closed if `DB.Querier()` fails.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Invert error handling in `DB.Querier()` and `DB.blockChunkQuerierForRange()` to make it clearer

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Ensure that queries that touch OOO data can't block OOO head garbage collection forever.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Address PR feedback: fix parameter name in comment

Co-authored-by: Jesus Vazquez <jesusvazquez@users.noreply.github.com>
Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com>

* Address PR feedback: use `lastGarbageCollectedMmapRef`

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Address PR feedback: ensure pending reads are cleaned up if creating an OOO querier fails

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com>
Co-authored-by: Jesus Vazquez <jesusvazquez@users.noreply.github.com>
2023-11-24 12:38:38 +01:00
Filip Petkovski 35a15e8f04
Add benchmark for native histograms (#13160)
* Add benchmark for native histograms

This commit adds a PromQL benchmark for queries on native histograms.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-11-23 14:09:17 +00:00
Paulin Todev 0102425af1
Use only one scrapeMetrics object per test. (#13051)
The scrape loop and scrape cache should use the same instance.
This brings the tests' behavior more in line with production.

Signed-off-by: Paulin Todev <paulin.todev@gmail.com>
2023-11-23 11:24:08 +00:00
Bryan Boreham 9051100aba Scraping: share buffer pool across all scrapes
Previously we had one per scrapePool, and one of those per configured
scraping job. Each pool holds a few unused buffers, so sharing one
across all scrapePools reduces total heap memory.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-23 10:23:34 +00:00