13525 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Manik Rana	8f088845d1	chore: use both scrape protocols in config ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-31 09:53:39 +05:30
Manik Rana	5f7bcff912	fix: check correct test cases ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-30 16:58:14 +05:30
Manik Rana	c58a173382	tests: add cases for OM text ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-30 16:57:06 +05:30
Manik Rana	7905fbbae0	chore: merge ct tests ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-30 16:49:17 +05:30
Manik Rana	7a3daa80a7	wip: add tests for OM CR parse ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-30 15:05:10 +05:30
Manik Rana	18f95cc994	Merge branch 'main' into implement-ct-in-append	2024-08-30 13:46:03 +05:30
Callum Styan	a77f5007f9	fix bug with metadata for rw2 (#14766) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	2024-08-30 08:14:20 +01:00
Arthur Silva Sens	ee2b7fdf9d	Merge pull request #14760 from bboreham/simplify-otel-time-conversion ... [REFACTOR] OTLP translator: simplify time conversion	2024-08-29 17:39:43 -03:00
Neeraj Gartia	8c7bf39d96	Moves TestNativeHistogram_MulDivOperator to promql testing framework (#14688) ... PromQL: add test for mul and div operator Also, remove the converted test from the engine_test.go file. This also includes an extension of the test framework to allow NaN/Inf in histogram buckets. --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2024-08-29 16:42:35 +02:00
Jorge Creixell	e9e3d64b7c	PromQL engine: Delay deletion of __name__ label to the end of the query evaluation (#14477) ... PromQL engine: Delay deletion of __name__ label to the end of the query evaluation - This change allows optionally preserving the `__name__` label via the `label_replace` and `label_join` functions, and helps prevent the dreaded "vector cannot contain metrics with the same labelset" error. - The implementation extends the `Series` and `Sample` structs with a boolean flag indicating whether the `__name__` label should be deleted at the end of the query evaluation. - The `label_replace` and `label_join` functions can still access the value of the `__name__` label, even if it has been previously marked for deletion. If `__name__` is used as target label, it won't be dropped at the end of the query evaluation. - Fixes https://github.com/prometheus/prometheus/issues/11397 - See https://github.com/jcreixell/prometheus/pull/2 for previous discussion, including the decision to create this PR and benchmark it before considering other alternatives (like refactoring `labels.Labels`). - See https://github.com/jcreixell/prometheus/pull/1 for an alternative implementation using a special label instead of boolean flags. - Note: a feature flag `promql-delayed-name-removal` has been added as it changes the behavior of some "weird" queries (see https://github.com/prometheus/prometheus/issues/11397#issuecomment-1451998792) Example (this always fails, as `__name__` is being dropped by `count_over_time`): ``` count_over_time({__name__!=""}[1m]) => Error executing query: vector cannot contain metrics with the same labelset ``` Before: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => Error executing query: vector cannot contain metrics with the same labelset ``` After: ``` label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)") => count_go_gc_cycles_automatic_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 count_go_gc_cycles_forced_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1 ... ``` Signed-off-by: Jorge Creixell <jcreixell@gmail.com> --------- Signed-off-by: Jorge Creixell <jcreixell@gmail.com> Signed-off-by: Björn Rabenstein <github@rabenste.in>	2024-08-29 15:50:39 +02:00
Bryan Boreham	5359567264	Merge pull request #14729 from bboreham/multimeta ... TSDB: streamline reading of overlapping head chunks	2024-08-29 14:11:45 +01:00
Bryan Boreham	f90c7a11d1	[REFACTOR] OTLP translator: simplify time conversion ... We don't need multiple levels of abstraction to convert nanoseconds to milliseconds. We do benefit from tests, however. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-29 12:00:01 +01:00
Bryan Boreham	1f38ae7bca	[TESTS] TSDB: fix up OOO tests for new Series behaviour ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-29 10:59:09 +01:00
Bryan Boreham	cde42f30e9	TSDB: streamline reading of overlapping head chunks ... `getOOOSeriesChunks` was already finding sets of overlapping chunks; we store those in a `multiMeta` struct so that `ChunkOrIterable` can reconstruct an `Iterable` easily and predictably. We no longer need a `MergeOOO` flag to indicate that this Meta should be merged with other ones; this is explicit in the `multiMeta` structure. We also no longer need `chunkMetaAndChunkDiskMapperRef`. Add `wrapOOOHeadChunk` to defeat `chunkenc.Pool` - chunks are reset during compaction, but if we wrap them (like `safeHeadChunk` was doing then this is skipped) . Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-29 10:57:29 +01:00
Bryan Boreham	838e49e7b8	[REFACTOR] TSDB: move chunkFromSeries from headChunkReader to head ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-29 10:51:48 +01:00
Arthur Silva Sens	5bd8988637	Simplify 'TestManagerCTZeroIngestion' (#14756) ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-08-29 08:40:17 +01:00
machine424	c586c15ae6	fix(discovery): make discovery manager notify consumers of dropped targets for still defined jobs ... scrape/manager_test.go: add a test to check that the manager gets notified for targets that got dropped by discovery to reproduce: https://github.com/prometheus/prometheus/issues/12858#issuecomment-1732318102 Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-08-28 17:39:02 +02:00
Björn Rabenstein	849215d90c	Merge pull request #14585 from fatsheep9146/covert-TestNativeHistogram_Sum_Count_Add_AvgOperator-to-framework ... convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework	2024-08-28 17:21:32 +02:00
Owen Williams	9da75328ea	fix(utf8): ensure correct validation when legacy mode turned on (#14736) ... fix(utf8): ensure correct validation when legacy mode turned on This depends on the included update of the prometheus/common dependency. --------- Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-28 17:15:42 +02:00
Björn Rabenstein	1d6e0071b7	Merge pull request #14751 from riskrole/main ... chore: fix some comments	2024-08-28 16:38:39 +02:00
Bryan Boreham	84f7e291e2	Merge pull request #14731 from pracucci/revert-14525-merge-mmmaxtime-into-shardhash ... Revert " Store `mmMaxTime` in same field as `seriesShard`"	2024-08-28 15:14:44 +01:00
Justin Lei	3a82cd5a7e	Add streaming remote read to ReadClient (#11379) ... * Add streaming remote read to ReadClient Signed-off-by: Justin Lei <justin.lei@grafana.com> * Apply suggestions from code review Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Justin Lei <justin.lei@grafana.com> * Remote read instrumentation tweaks Signed-off-by: Justin Lei <lei.justin@gmail.com> * Minor cleanups Signed-off-by: Justin Lei <lei.justin@gmail.com> * In-line handleChunkedResponse Signed-off-by: Justin Lei <lei.justin@gmail.com> * Fix lints Signed-off-by: Justin Lei <lei.justin@gmail.com> * Explicitly call cancel() when needed Signed-off-by: Justin Lei <lei.justin@gmail.com> * Update chunkedSeries, chunkedSeriesIterator for new interfaces Signed-off-by: Justin Lei <lei.justin@gmail.com> * Adapt remote.chunkedSeries to use prompb.ChunkedSeries Signed-off-by: Justin Lei <lei.justin@gmail.com> * Fix lint Signed-off-by: Justin Lei <lei.justin@gmail.com> --------- Signed-off-by: Justin Lei <justin.lei@grafana.com> Signed-off-by: Justin Lei <lei.justin@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	2024-08-28 07:23:54 +01:00
riskrole	406bf775aa	chore: fix some comments ... Signed-off-by: riskrole <yuhang@before.tech>	2024-08-28 11:26:57 +08:00
Suraj Patil	7757794bb3	[ENHANCEMENT] Promtool: Adding labels to time series while creating tsdb blocks (#14403) ... * feat: #14402 - Adding labels to time series while creating tsdb blocks Signed-off-by: Suraj Patil <patilsuraj767@gmail.com>	2024-08-28 12:12:24 +10:00
Bryan Boreham	9a813d5d0f	Merge pull request #14741 from prometheus/merge-2.54.1-into-main ... Merge 2.54.1 into main	2024-08-27 17:58:17 +01:00
Manik Rana	bec708eda8	chore: gofumpt ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-27 21:59:22 +05:30
Manik Rana	fd2a2b1b9e	fix: set skipOMSeries in test ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-27 21:10:51 +05:30
Manik Rana	0caaa96206	chore: use textparse interface to set om options ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-27 21:06:40 +05:30
Manik Rana	3de6bffae7	chore: lint ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-27 21:04:07 +05:30
Manik Rana	d14eb733e8	refac: allow opt-in to OM CT ingestion ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-27 21:01:21 +05:30
Bryan Boreham	4202be5e79	Merge branch 'release-2.54' into merge-2.54.1-into-main	2024-08-27 12:04:48 +01:00
Bryan Boreham	e6cfa720fb	Merge pull request #14739 from prometheus/prep-2.54.1 ... Cut release 2.54.1	2024-08-27 11:44:00 +01:00
Bryan Boreham	da28f88910	Cut release 2.54.1 ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-27 10:14:16 +01:00
Manik Rana	82064d42b8	feat: init NewOpenMetricsParser with skipCT true ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-27 02:32:25 +05:30
Manik Rana	c4052ba23d	chore: revert TypeRequiresCT to private ... Signed-off-by: Manik Rana <manikrana54@gmail.com>	2024-08-27 02:31:47 +05:30
Ganesh Vernekar	b09eaf8acd	Merge pull request #14727 from prometheus/owilliams/names-config-fix ... fix(utf8): fix config logic for name validation	2024-08-26 12:57:26 -04:00
Julien	349068ad3e	Merge pull request #14665 from roidelapluie/multiple-listening-addresses ... Add support for multiple listening addresses	2024-08-26 10:31:23 +02:00
Marco Pracucci	ef649d5968	Revert " Store mmMaxTime in same field as seriesShard" ... Signed-off-by: Marco Pracucci <marco@pracucci.com>	2024-08-26 08:56:16 +02:00
Devin Trejo	d4994e5bc4	fix: Remote-write-reciever returns 4xx when request contains a time series with duplicate labels. (#14716) ... Signed-off-by: Devin Trejo <dtrejo@palantir.com>	2024-08-23 15:15:27 -07:00
Owen Williams	436a439ed2	fix(utf8): fix config logic for name validation ... We should only overwrite the ScrapeConfig if it is empty. Added tests part of https://github.com/prometheus/prometheus/issues/13095 Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-23 15:16:48 -04:00
George Krajcsovits	1b86d54c7f	Merge pull request #14714 from krajorama/fix-ooo-chunkoriterablewithcopy ... tsdb: Make requesting merge with OOO head explicit in chunk.Meta	2024-08-23 16:29:07 +02:00
György Krajcsovits	183bbc39a2	Make requesting merge with OOO head explicit in chunk.Meta ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-23 15:50:53 +02:00
György Krajcsovits	41c076196e	New cases in Test_ChunkQuerier_OOOQuery and Test_Querier_OOOQuery ... Case 1: OOO in-memory head chunk overlaps with first mmaped in-order chunk. Query: |----------------------------------------------------------------| InO: |------mmap---------------||---------mem----------------------| OOO: |-----mem-----------| This triggers ChunkOrIterableWithCopy not including OOO head chunks bug. Similar to #14693 however testing the end of the interval doesn't trigger the problem because there the in-order head chunk will be trimmed with a tombstone, causing the code to switch to ChunkOrIterable which was fixed. See a36d1a8a92/tsdb/querier.go (L646) where len(p.bufIter.Intervals) will be non zero, because it includes the tombstone to trim the result to the query max time. Thus a new test is added to check the overlap at the beginning of the interval that has a separate chunk, which does not need trimming. Note: same test doesn't fail for sample querier in Test_Querier_OOOQuery as that doesn't use copy, that is copyHeadChunk is false in the if condition above. Case 2: OOO mmaped head chunk overlaps with first mmaped in-order chunk. Query: |----------------------------------------------------------------| InO: |------mmap---------------||---------mem----------------------| OOO: |-----mmap-----------| |--mem--| In this case the meta contains the reference of the in-order chunk and no indication that a merge is needed with the OOO mmaped chunk. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-23 15:50:47 +02:00
Bryan Boreham	0538ad3a08	Merge pull request #14719 from aknuds1/arve/fix-valnone ... [COMMENT] tsdb: Fix ValNone typo	2024-08-23 09:32:59 +01:00
Arve Knudsen	b0aba26ed5	tsdb: Fix ValNone typo in comment ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-23 08:20:20 +02:00
Ziqi Zhao	87eab0aaad	fix golang lint ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-23 09:32:04 +08:00
Ziqi Zhao	c7c4a5c347	add helper function to compare native histograms in testing ... Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>	2024-08-23 09:28:31 +08:00
Björn Rabenstein	5fd2717aec	Merge pull request #14715 from prometheus/beorn7/lint ... lint: Revamp our linting rules, mostly around doc comments	2024-08-22 17:56:40 +02:00
beorn7	0f760f63dd	lint: Revamp our linting rules, mostly around doc comments ... Several things done here: - Set `max-issues-per-linter` to 0 so that we actually see all linter warnings and not just 50 per linter. (As we also set `max-same-issues` to 0, I assume this was the intention from the beginning.) - Stop using the golangci-lint default excludes (by setting `exclude-use-default: false`. Those are too generous and don't match our style conventions. (I have re-added some of the excludes explicitly in this commit. See below.) - Re-add the `errcheck` exclusion we have used so far via the defaults. - Exclude the signature requirement `govet` has for `Seek` methods because we use non-standard `Seek` methods a lot. (But we keep other requirements, while the default excludes completely disabled the check for common method segnatures.) - Exclude warnings about missing doc comments on exported symbols. (We used to be pretty adamant about doc comments, but stopped that at some point in the past. By now, we have about 500 missing doc comments. We may consider reintroducing this check, but that's outside of the scope of this commit. The default excludes of golangci-lint essentially ignore doc comments completely.) - By stop using the default excludes, we now get warnings back on malformed doc comments. That's the most impactful change in this commit. It does not enforce doc comments (again), but _if_ there is a doc comment, it has to have the recommended form. (Most of the changes in this commit are fixing this form.) - Improve wording/spelling of some comments in .golangci.yml, and remove an outdated comment. - Leave `package-comments` inactive, but add a TODO asking if we should change that. - Add a new sub-linter `comment-spacings` (and fix corresponding comments), which avoids missing spaces after the leading `//`. Signed-off-by: beorn7 <beorn@grafana.com>	2024-08-22 17:36:11 +02:00
Bryan Boreham	95b3c49836	[TESTS] remote-write: fix slice init length (#14702) ... fix: fix slice init length	2024-08-22 12:53:45 +01:00