Commit graph

11726 commits

Author SHA1 Message Date
Jeanette Tan 3ccaaa40ba Fix according to code review
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:37:07 +08:00
György Krajcsovits 4296ecbd14 tsdb/compact_test.go: test mixed typed series with PopulateBlock
Add testcase and update test so that it can test native histograms as well.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-11-02 13:33:42 +08:00
Jeanette Tan 27abf09e7f Fix missing MinTime in histogram chunks
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:39 +08:00
Jeanette Tan 2f7060bd5a Expand TestPopulateWithTombSeriesIterators to test earlier deletion intervals for histogram chunks as well as time-overlapping chunks
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:35 +08:00
Jeanette Tan 7a4a1127b7 Expand TestPopulateWithTombSeriesIterators to test min max times of chunks, including mixed chunks
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:33 +08:00
Jeanette Tan 04aabdd7cc Refactor TestPopulateWithDelSeriesIterator unit tests to reuse more code
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:30 +08:00
Jeanette Tan 46be85f2dc Make TestPopulateWithDelSeriesIterator tests cover histogram types and check MinTime
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:26 +08:00
Björn Rabenstein 96179fe06a
Merge pull request #13055 from prometheus/beorn7/storage
storage: Fix mixed samples handling in sampleRing
2023-10-31 17:10:24 +01:00
beorn7 4696b46dd5 storage: Fix mixed samples handling in sampleRing
Two issues are fixed here, that lead to the same problem:

1. If `newSampleRing` is called with an unknown ValueType including
   ValueNone, we have initialized the interface buffer (`iBuf`).
   However, we would still use a specialized buffer for the first
   sample, opportunistically assuming that we might still not
   encounter mixed samples and we should go down the more efficient
   road.

2. If the `sampleRing` is `reset`, we leave all buffers alone,
   including `iBuf`, which is generally fine, but not for `iBuf`, see
   below.

In both cases, `iBuf` already contains values, but we will fill one of
the specialized buffers first. Once we then actually encounter mixed
samples, the content of the specialized buffer is copied into `iBuf`
using `append`. That's by itself the right idea because `iBuf` might
be `nil`, and even if not, it might or might not have the right
capacity. However, this approach assumes that `iBuf` is empty, or more
precisely has a length of zero.

This commit makes sure that `iBuf` does not get needlessly initialized
in `newSampleRing` and that it is emptied upon `reset`.

A test case is added to demonstrate both issues above.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-10-31 16:18:09 +01:00
Levi Harrison 8db8ad1bae
bump version (#13032)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-27 23:25:56 +02:00
Levi Harrison 7bdabb01d2
Release 2.48.0-rc.1 (#13028)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-25 12:33:03 +01:00
zenador 80e977aae6
Remove NewPossibleNonCounterInfo and minimise creating empty annotations (#13012)
* Remove NewPossibleNonCounterInfo until it can be made more efficient, and avoid creating empty annotations as much as possible

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-10-24 17:36:07 +01:00
Bryan Boreham 551fa8378c
Merge pull request #12959 from LeviHarrison/release-2.48.0-rc0
Release 2.48.0-rc0
2023-10-17 15:06:40 +01:00
Levi Harrison d1620abde9 Add last warning pr
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-17 09:37:50 -04:00
Björn Rabenstein fb0b9fc0da
Merge pull request #12988 from zenador/release-2.48-fix-non-counter-warning
Fix possible non-counter warning for empty names and native histograms
2023-10-17 11:41:20 +02:00
Levi Harrison 700f9bd7c6 nits
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-16 22:26:45 -04:00
Jeanette Tan 9a8bd8eac6 Fix possible non-counter warning for empty names and native histograms
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-10-16 15:52:10 +08:00
Levi Harrison 827804c79d Release 2.48.0-rc0
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 22:32:41 -04:00
Julien Pivotto 8a0c9aec1c
Merge pull request #12985 from LeviHarrison/merge-main-release-2.48
Cherry pick latest main changes into 2.48
2023-10-15 22:28:34 +02:00
Levi Harrison 454a0a2c1b Update dependencies for 2.48 (#12964)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Julius Volz 191c24a0ed Fix: Exempt "_bucket" suffix from PossibleNonCounterInfo warning (#12982)
Related to PR #12152

Signed-off-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Julien Pivotto 77c1c2c44a Merge pull request #12981 from prometheus/dependabot/go_modules/documentation/examples/remote_storage/github.com/prometheus/prometheus-0.47.2
build(deps): bump github.com/prometheus/prometheus from 0.45.0 to 0.47.2 in /documentation/examples/remote_storage

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Ziqi Zhao 60e2bb0135 enhance promtool tsdb analyze command (#12869)
Improve promtool tsdb analyze

- Make it more suitable for variable size float chunks.
- Add support for histogram chunks.

---------

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Levi Harrison 4284e6f2f7 Merge pull request #11860 from mmorel-35/azure-sdk-for-go
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Paschalis Tsilias 42b8f2f5fc tsdb/agent: allow ingestion of OOO samples (#12897)
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Ganesh Vernekar 4df2f2432b Additionally wrap WBL replay error (#12406)
* Additionally wrap WBL replay error

Although WBL replay is already wrapped with errLoadWbl,
there are other errors that can happen during a WBL replay.
We should not try to repair WAL in those cases.

This commit additionally wraps the final error in Head.Init again
with errLoadWbl so that WBL replay errors can be identified properly.

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>
Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Levi Harrison 4307fa19ee Merge pull request #12819 from prometheus/dependabot/go_modules/golang.org/x/net-0.15.0
build(deps): bump golang.org/x/net from 0.13.0 to 0.15.0

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Levi Harrison 7f7cb471b8 Merge pull request #12848 from prometheus/dependabot/go_modules/k8s-io-815bab050d
build(deps): bump the k8s-io group with 2 updates

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Levi Harrison 281dc29677 Merge pull request #12912 from prometheus/dependabot/go_modules/github.com/aws/aws-sdk-go-1.45.19
build(deps): bump github.com/aws/aws-sdk-go from 1.44.317 to 1.45.19

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Arthur Silva Sens 644cb3be70 Add created timestamps to prompb (#12936)
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Levi Harrison ea3ad744a9
Merge pull request #12956 from LeviHarrison/shepherd-2.48 2023-10-09 13:01:57 -04:00
Bryan Boreham f6d9c84fde
scraping: delay creating buffer, to save memory (#12953)
We don't need the buffer to read the response until the scrape http call
returns; creating it earlier makes the buffer pool larger.

I split `scrape()` into `scrape()` which returns with the http response,
and `readResponse()` which decompresses and copies the data into the
supplied buffer. This design was chosen to minimize impact on the logic.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-10-09 17:23:53 +01:00
Levi Harrison e7a1d3bfaf Volunteer Levi Harrison for release shepherd
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-09 12:21:57 -04:00
Oleg Zaytsev fe90dcccff
Revert ListPostings change (#12955)
Reverts change from https://github.com/prometheus/prometheus/pull/12906

The benchmarks show that it's slower when intersecting, which is a
common usage for ListPostings (when intersecting matchers from Head)

(old is before #12906, new is #12906):

                           │     old     │                 new                 │
                           │   sec/op    │   sec/op     vs base                │
Intersect/LongPostings1-16   20.54µ ± 1%   21.11µ ± 1%   +2.76% (p=0.000 n=20)
Intersect/LongPostings2-16   51.03m ± 1%   52.40m ± 2%   +2.69% (p=0.000 n=20)
Intersect/ManyPostings-16    194.2m ± 3%   332.1m ± 1%  +71.00% (p=0.000 n=20)
geomean                      5.882m        7.161m       +21.74%

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2023-10-09 17:25:18 +02:00
Oleg Zaytsev 5bd8c8c561
Clarify Postings.At() contract (#12921)
It's implicit, but should be explicit. It is invalid to call At() after
a failed call to Next() or Seek().

Following up on https://github.com/prometheus/prometheus/pull/12906

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2023-10-09 16:15:06 +02:00
Björn Rabenstein b787e5f190
Merge pull request #12931 from zenador/warning-non-monotonic-classic-histogram
Add warning when monotonicity is forced in the input to histogram_quantile
2023-10-08 15:59:48 +02:00
Danny Kopping d3554d8421
Show group interval in Rules display (#12943)
* Show group interval in Rules display

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

* Humanise interval

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

---------

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
2023-10-08 14:51:24 +02:00
Bryan Boreham a5a4eab679
Storage: reduce memory allocations when merging series sets (#12938)
Instead of setting to nil and allocating a new slice every time the
merge is advanced, re-use the previous slice.
This is safe because the `currentSets` member is only used inside member
functions, and explicitly copied in `At()`, the only place it leaves the
struct.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-10-06 12:28:07 +01:00
Bryan Boreham b8f8c3c277
Merge pull request #12939 from bboreham/hoist-scrape-labels
scraping: hoist labels variable to save garbage
2023-10-06 12:27:32 +01:00
Jeanette Tan 0cbf0c1c68 Revise according to code review
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-10-06 19:09:32 +08:00
Bryan Boreham 7c934ae18c scraping: hoist labels variable to save garbage
`lset` escapes to heap due to being passed through the text-parser
interface, so we can reduce garbage by hoisting it out of the loop so
only one allocation is done for every series in a scrape.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-10-05 11:04:59 +00:00
rakshith210 cdad64002a
Added Azure OAuth support (#12572)
* Added Azure OAuth support

Signed-off-by: rakshith210 <rakshith.me@gmail.com>

* Added missing comment

Signed-off-by: rakshith210 <rakshith.me@gmail.com>

* Addressing comment

Signed-off-by: rakshith210 <rakshith.me@gmail.com>

* Fixed lint issue

Signed-off-by: rakshith210 <rakshith.me@gmail.com>

* Fix test

Signed-off-by: rakshith210 <rakshith.me@gmail.com>

* Addressing comments

Signed-off-by: rakshith210 <rakshith.me@gmail.com>

* Added documentation and updated unit tests

Signed-off-by: rakshith210 <rakshith.me@gmail.com>

* Addressing comments

Signed-off-by: rakshith210 <rakshith.me@gmail.com>

---------

Signed-off-by: rakshith210 <rakshith.me@gmail.com>
2023-10-04 22:16:36 -04:00
Julien Pivotto 0331bcc7c9
Merge pull request #12932 from prometheus/release-2.47
Merge release 2.47 into main
2023-10-04 15:05:53 +02:00
Jeanette Tan feaa93da77 Add warning when monotonicity is forced in the input to histogram_quantile
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-10-04 18:53:55 +08:00
Bryan Boreham c4d1a8beff
Merge pull request #12930 from prometheus/superq/pick-12874
Release 2.47.1
2023-10-04 11:06:28 +01:00
Arve Knudsen b43358fd43
API: Add tests for query timeout parameter (#12927)
Add unit tests for Web API's query endpoint (GET/POST).
Also modify the endpoint handler to use context.WithDeadline instead of
context.WithTimeout, so the deadline is deterministic for the tests.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-10-04 10:36:55 +02:00
SuperQ 64130d7909
Release 2.47.1
* [BUGFIX] Fix duplicate sample detection at chunk size limit #12874

Signed-off-by: SuperQ <superq@gmail.com>
2023-10-04 09:46:57 +02:00
Björn Rabenstein ec9170b8bf
Merge pull request #12874 from krajorama/outof-order-chunks
Fix duplicate sample detection at chunk size limit
Signed-off-by: SuperQ <superq@gmail.com>
2023-10-04 09:44:34 +02:00
gotjosh 79be1b8357
Merge pull request #12909 from 09jvilla/patch-1
Clarify what happens when a rule group takes too long to execute
2023-10-03 10:43:57 +01:00
Jennifer Villa ea7bec4cc7
Update recording_rules.md
updated language to be a bit more clear

Signed-off-by: Jennifer Villa <jvilla2013@gmail.com>
2023-10-02 21:44:54 -04:00