13509 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Neeraj Gartia	8c7bf39d96	Moves TestNativeHistogram_MulDivOperator to promql testing framework (#14688) ... PromQL: add test for mul and div operator Also, remove the converted test from the engine_test.go file. This also includes an extension of the test framework to allow NaN/Inf in histogram buckets. --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-08-29 16:42:35 +02:00
Jorge Creixell	e9e3d64b7c	PromQL engine: Delay deletion of __name__ label to the end of the query evaluation (#14477) ... PromQL engine: Delay deletion of __name__ label to the end of the query evaluation - This change allows optionally preserving the `__name__` label via the `label_replace` and `label_join` functions, and helps prevent the dreaded "vector cannot contain metrics with the same labelset" error. - The implementation extends the `Series` and `Sample` structs with a boolean flag indicating whether the `__name__` label should be deleted at the end of the query evaluation. - The `label_replace` and `label_join` functions can still access the value of the `__name__` label, even if it has been previously marked for deletion. If `__name__` is used as target label, it won't be dropped at the end of the query evaluation. - Fixes https://github.com/prometheus/prometheus/issues/11397 - See https://github.com/jcreixell/prometheus/pull/2 for previous discussion, including the decision to create this PR and benchmark it before considering other alternatives (like refactoring `labels.Labels`). - See https://github.com/jcreixell/prometheus/pull/1 for an alternative implementation using a special label instead of boolean flags. - Note: a feature flag `promql-delayed-name-removal` has been added as it changes the behavior of some "weird" queries (see https://github.com/prometheus/prometheus/issues/11397#issuecomment-1451998792) Example (this always fails, as `__name__` is being dropped by `count_over_time`): ``` count_over_time({__name__!=""}[1m]) => Error executing query: vector cannot contain metrics with the same labelset ``` Before: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => Error executing query: vector cannot contain metrics with the same labelset ``` After: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => count_go_gc_cycles_automatic_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 count_go_gc_cycles_forced_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 ... ``` Signed-off-by: Jorge Creixell <jcreixell@gmail.com> --------- Signed-off-by: Jorge Creixell <jcreixell@gmail.com> Signed-off-by: Björn Rabenstein <github@rabenste.in>	2024-08-29 15:50:39 +02:00
Bryan Boreham	5359567264	Merge pull request #14729 from bboreham/multimeta ... TSDB: streamline reading of overlapping head chunks	2024-08-29 14:11:45 +01:00
Bryan Boreham	1f38ae7bca	[TESTS] TSDB: fix up OOO tests for new Series behaviour ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-29 10:59:09 +01:00
Bryan Boreham	cde42f30e9	TSDB: streamline reading of overlapping head chunks ... `getOOOSeriesChunks` was already finding sets of overlapping chunks; we store those in a `multiMeta` struct so that `ChunkOrIterable` can reconstruct an `Iterable` easily and predictably. We no longer need a `MergeOOO` flag to indicate that this Meta should be merged with other ones; this is explicit in the `multiMeta` structure. We also no longer need `chunkMetaAndChunkDiskMapperRef`. Add `wrapOOOHeadChunk` to defeat `chunkenc.Pool` - chunks are reset during compaction, but if we wrap them (like `safeHeadChunk` was doing then this is skipped) . Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-29 10:57:29 +01:00
Bryan Boreham	838e49e7b8	[REFACTOR] TSDB: move chunkFromSeries from headChunkReader to head ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-29 10:51:48 +01:00
Arthur Silva Sens	5bd8988637	Simplify 'TestManagerCTZeroIngestion' (#14756) ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-08-29 08:40:17 +01:00
machine424	c586c15ae6	fix(discovery): make discovery manager notify consumers of dropped targets for still defined jobs ... scrape/manager_test.go: add a test to check that the manager gets notified for targets that got dropped by discovery to reproduce: https://github.com/prometheus/prometheus/issues/12858#issuecomment-1732318102 Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-08-28 17:39:02 +02:00
Björn Rabenstein	849215d90c	Merge pull request #14585 from fatsheep9146/covert-TestNativeHistogram_Sum_Count_Add_AvgOperator-to-framework ... convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework	2024-08-28 17:21:32 +02:00
Owen Williams	9da75328ea	fix(utf8): ensure correct validation when legacy mode turned on (#14736) ... fix(utf8): ensure correct validation when legacy mode turned on This depends on the included update of the prometheus/common dependency. --------- Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-28 17:15:42 +02:00
Björn Rabenstein	1d6e0071b7	Merge pull request #14751 from riskrole/main ... chore: fix some comments	2024-08-28 16:38:39 +02:00
Bryan Boreham	84f7e291e2	Merge pull request #14731 from pracucci/revert-14525-merge-mmmaxtime-into-shardhash ... Revert " Store `mmMaxTime` in same field as `seriesShard`"	2024-08-28 15:14:44 +01:00
Justin Lei	3a82cd5a7e	Add streaming remote read to ReadClient (#11379) ... * Add streaming remote read to ReadClient Signed-off-by: Justin Lei <justin.lei@grafana.com> * Apply suggestions from code review Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Justin Lei <justin.lei@grafana.com> * Remote read instrumentation tweaks Signed-off-by: Justin Lei <lei.justin@gmail.com> * Minor cleanups Signed-off-by: Justin Lei <lei.justin@gmail.com> * In-line handleChunkedResponse Signed-off-by: Justin Lei <lei.justin@gmail.com> * Fix lints Signed-off-by: Justin Lei <lei.justin@gmail.com> * Explicitly call cancel() when needed Signed-off-by: Justin Lei <lei.justin@gmail.com> * Update chunkedSeries, chunkedSeriesIterator for new interfaces Signed-off-by: Justin Lei <lei.justin@gmail.com> * Adapt remote.chunkedSeries to use prompb.ChunkedSeries Signed-off-by: Justin Lei <lei.justin@gmail.com> * Fix lint Signed-off-by: Justin Lei <lei.justin@gmail.com> --------- Signed-off-by: Justin Lei <justin.lei@grafana.com> Signed-off-by: Justin Lei <lei.justin@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	2024-08-28 07:23:54 +01:00
riskrole	406bf775aa	chore: fix some comments ... Signed-off-by: riskrole <yuhang@before.tech>	2024-08-28 11:26:57 +08:00
Suraj Patil	7757794bb3	[ENHANCEMENT] Promtool: Adding labels to time series while creating tsdb blocks (#14403) ... * feat: #14402 - Adding labels to time series while creating tsdb blocks Signed-off-by: Suraj Patil <patilsuraj767@gmail.com>	2024-08-28 12:12:24 +10:00
Bryan Boreham	9a813d5d0f	Merge pull request #14741 from prometheus/merge-2.54.1-into-main ... Merge 2.54.1 into main	2024-08-27 17:58:17 +01:00
Bryan Boreham	4202be5e79	Merge branch 'release-2.54' into merge-2.54.1-into-main	2024-08-27 12:04:48 +01:00
Bryan Boreham	e6cfa720fb	Merge pull request #14739 from prometheus/prep-2.54.1 ... Cut release 2.54.1	2024-08-27 11:44:00 +01:00
Bryan Boreham	da28f88910	Cut release 2.54.1 ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-27 10:14:16 +01:00
Ganesh Vernekar	b09eaf8acd	Merge pull request #14727 from prometheus/owilliams/names-config-fix ... fix(utf8): fix config logic for name validation	2024-08-26 12:57:26 -04:00
Julien	349068ad3e	Merge pull request #14665 from roidelapluie/multiple-listening-addresses ... Add support for multiple listening addresses	2024-08-26 10:31:23 +02:00
Marco Pracucci	ef649d5968	Revert " Store mmMaxTime in same field as seriesShard" ... Signed-off-by: Marco Pracucci <marco@pracucci.com>	2024-08-26 08:56:16 +02:00
Devin Trejo	d4994e5bc4	fix: Remote-write-reciever returns 4xx when request contains a time series with duplicate labels. (#14716) ... Signed-off-by: Devin Trejo <dtrejo@palantir.com>	2024-08-23 15:15:27 -07:00
Owen Williams	436a439ed2	fix(utf8): fix config logic for name validation ... We should only overwrite the ScrapeConfig if it is empty. Added tests part of https://github.com/prometheus/prometheus/issues/13095 Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-23 15:16:48 -04:00
George Krajcsovits	1b86d54c7f	Merge pull request #14714 from krajorama/fix-ooo-chunkoriterablewithcopy ... tsdb: Make requesting merge with OOO head explicit in chunk.Meta	2024-08-23 16:29:07 +02:00
György Krajcsovits	183bbc39a2	Make requesting merge with OOO head explicit in chunk.Meta ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-23 15:50:53 +02:00
György Krajcsovits	41c076196e	New cases in Test_ChunkQuerier_OOOQuery and Test_Querier_OOOQuery ... Case 1: OOO in-memory head chunk overlaps with first mmaped in-order chunk. Query: |----------------------------------------------------------------| InO: |------mmap---------------||---------mem----------------------| OOO: |-----mem-----------| This triggers ChunkOrIterableWithCopy not including OOO head chunks bug. Similar to #14693 however testing the end of the interval doesn't trigger the problem because there the in-order head chunk will be trimmed with a tombstone, causing the code to switch to ChunkOrIterable which was fixed. See a36d1a8a92/tsdb/querier.go (L646) where len(p.bufIter.Intervals) will be non zero, because it includes the tombstone to trim the result to the query max time. Thus a new test is added to check the overlap at the beginning of the interval that has a separate chunk, which does not need trimming. Note: same test doesn't fail for sample querier in Test_Querier_OOOQuery as that doesn't use copy, that is copyHeadChunk is false in the if condition above. Case 2: OOO mmaped head chunk overlaps with first mmaped in-order chunk. Query: |----------------------------------------------------------------| InO: |------mmap---------------||---------mem----------------------| OOO: |-----mmap-----------| |--mem--| In this case the meta contains the reference of the in-order chunk and no indication that a merge is needed with the OOO mmaped chunk. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-23 15:50:47 +02:00
Bryan Boreham	0538ad3a08	Merge pull request #14719 from aknuds1/arve/fix-valnone ... [COMMENT] tsdb: Fix ValNone typo	2024-08-23 09:32:59 +01:00
Arve Knudsen	b0aba26ed5	tsdb: Fix ValNone typo in comment ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-23 08:20:20 +02:00
Ziqi Zhao	87eab0aaad	fix golang lint ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-23 09:32:04 +08:00
Ziqi Zhao	c7c4a5c347	add helper function to compare native histograms in testing ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-23 09:28:31 +08:00
Björn Rabenstein	5fd2717aec	Merge pull request #14715 from prometheus/beorn7/lint ... lint: Revamp our linting rules, mostly around doc comments	2024-08-22 17:56:40 +02:00
beorn7	0f760f63dd	lint: Revamp our linting rules, mostly around doc comments ... Several things done here: - Set `max-issues-per-linter` to 0 so that we actually see all linter warnings and not just 50 per linter. (As we also set `max-same-issues` to 0, I assume this was the intention from the beginning.) - Stop using the golangci-lint default excludes (by setting `exclude-use-default: false`. Those are too generous and don't match our style conventions. (I have re-added some of the excludes explicitly in this commit. See below.) - Re-add the `errcheck` exclusion we have used so far via the defaults. - Exclude the signature requirement `govet` has for `Seek` methods because we use non-standard `Seek` methods a lot. (But we keep other requirements, while the default excludes completely disabled the check for common method segnatures.) - Exclude warnings about missing doc comments on exported symbols. (We used to be pretty adamant about doc comments, but stopped that at some point in the past. By now, we have about 500 missing doc comments. We may consider reintroducing this check, but that's outside of the scope of this commit. The default excludes of golangci-lint essentially ignore doc comments completely.) - By stop using the default excludes, we now get warnings back on malformed doc comments. That's the most impactful change in this commit. It does not enforce doc comments (again), but _if_ there is a doc comment, it has to have the recommended form. (Most of the changes in this commit are fixing this form.) - Improve wording/spelling of some comments in .golangci.yml, and remove an outdated comment. - Leave `package-comments` inactive, but add a TODO asking if we should change that. - Add a new sub-linter `comment-spacings` (and fix corresponding comments), which avoids missing spaces after the leading `//`. Signed-off-by: beorn7 <beorn@grafana.com>	2024-08-22 17:36:11 +02:00
Bryan Boreham	95b3c49836	[TESTS] remote-write: fix slice init length (#14702) ... fix: fix slice init length	2024-08-22 12:53:45 +01:00
Björn Rabenstein	0a00d21cff	Merge pull request #14673 from machine424/docdoccc ... docs(configuration.md): clarify the explanations about some temp labe…	2024-08-22 12:31:46 +02:00
Arve Knudsen	a36d1a8a92	Merge pull request #14707 from aknuds1/arve/upgradegolangci-lint ... Upgrade golangci-lint to v1.60.2	2024-08-22 10:58:42 +02:00
Arve Knudsen	fbcd50f32c	Upgrade golangci-lint to v1.60.2 ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-21 18:57:17 +02:00
Arve Knudsen	7cfe0b1567	Merge pull request #14706 from aknuds1/arve/otlp-invalid-histograms ... OTLP receiver: Warn when encountering invalid exponential histograms	2024-08-21 18:07:45 +02:00
Arve Knudsen	b50c5d42fe	OTLP receiver: Warn when encountering invalid exponential histograms ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-21 16:43:16 +02:00
Bryan Boreham	9a74d53935	[BUGFIX] TSDB: Fix query overlapping in-order and ooo head (#14693) ... * tsdb: Unit test query overlapping in order and ooo head Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * TSDB: Merge overlapping head chunk The basic idea is that getOOOSeriesChunks can populate Meta.Chunk, but since it only returns one Meta per overlapping time-slot, that pointer may end up in a Meta with a head-chunk ID. So we need HeadAndOOOChunkReader.ChunkOrIterable() to call mergedChunks in that case. Previously, mergedChunks was checking that meta.Ref was a valid OOO chunk reference, but it never actually uses that reference; it just finds all chunks overlapping in time. So we can delete that code. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-21 14:24:20 +01:00
Björn Rabenstein	7fad1ec8ee	Merge pull request #14655 from suntala/suntala/sort-by-label-enhancement ... promql: Fall back to full label sets when sorting by label	2024-08-21 12:28:55 +02:00
cuishuang	2110661121	fix: fix slice init length ... Signed-off-by: cuishuang <imcusg@gmail.com>	2024-08-21 18:15:25 +08:00
Ziqi Zhao	8f828d45c1	convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-21 09:24:50 +08:00
suntala	8e82ac8d2b	Add comments to the sort by label functions ... Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-20 20:41:55 +02:00
Ben Kochie	1435c8ae4a	Include test CA text info (#14699) ... Use `openssl x509 -text -in <file>` to include the text version of the certificate in order to make it easier to see the diff when applying changes to the cert. Signed-off-by: SuperQ <superq@gmail.com>	2024-08-20 19:41:02 +02:00
Julien	f9f39a4411	Extend testing CA certificates (#14696) ... Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-08-20 17:09:07 +02:00
machine424	88cac6fb49	docs(configuration.md): clarify the explanations about some temp labels and format for better visibility. ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-08-20 15:44:39 +02:00
Julien	481d718539	Merge pull request #14661 from prymitive/TestHangingNotifier ... tests: increase TestHangingNotifier timeout	2024-08-20 14:37:16 +02:00
Julien Pivotto	9b5e7623f4	Add support for multiple listening addresses ... Fixes #9105 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-08-20 13:09:16 +02:00
Julien	82c4599ebe	Merge pull request #14690 from harshitasao/fix-vulnerability ... fix: fixed the vulnerability	2024-08-20 13:06:51 +02:00