13473 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Callum Styan	75e6497ec1	add a test that simulates the user having two remote writes configured, ... thus duplicating the stored/interned series labels Signed-off-by: Callum Styan <callumstyan@gmail.com>	2024-08-27 10:50:29 -07:00
Ganesh Vernekar	b09eaf8acd	Merge pull request #14727 from prometheus/owilliams/names-config-fix ... fix(utf8): fix config logic for name validation	2024-08-26 12:57:26 -04:00
Julien	349068ad3e	Merge pull request #14665 from roidelapluie/multiple-listening-addresses ... Add support for multiple listening addresses	2024-08-26 10:31:23 +02:00
Devin Trejo	d4994e5bc4	fix: Remote-write-reciever returns 4xx when request contains a time series with duplicate labels. (#14716) ... Signed-off-by: Devin Trejo <dtrejo@palantir.com>	2024-08-23 15:15:27 -07:00
Owen Williams	436a439ed2	fix(utf8): fix config logic for name validation ... We should only overwrite the ScrapeConfig if it is empty. Added tests part of https://github.com/prometheus/prometheus/issues/13095 Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-23 15:16:48 -04:00
George Krajcsovits	1b86d54c7f	Merge pull request #14714 from krajorama/fix-ooo-chunkoriterablewithcopy ... tsdb: Make requesting merge with OOO head explicit in chunk.Meta	2024-08-23 16:29:07 +02:00
György Krajcsovits	183bbc39a2	Make requesting merge with OOO head explicit in chunk.Meta ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-23 15:50:53 +02:00
György Krajcsovits	41c076196e	New cases in Test_ChunkQuerier_OOOQuery and Test_Querier_OOOQuery ... Case 1: OOO in-memory head chunk overlaps with first mmaped in-order chunk. Query: |----------------------------------------------------------------| InO: |------mmap---------------||---------mem----------------------| OOO: |-----mem-----------| This triggers ChunkOrIterableWithCopy not including OOO head chunks bug. Similar to #14693 however testing the end of the interval doesn't trigger the problem because there the in-order head chunk will be trimmed with a tombstone, causing the code to switch to ChunkOrIterable which was fixed. See a36d1a8a92/tsdb/querier.go (L646) where len(p.bufIter.Intervals) will be non zero, because it includes the tombstone to trim the result to the query max time. Thus a new test is added to check the overlap at the beginning of the interval that has a separate chunk, which does not need trimming. Note: same test doesn't fail for sample querier in Test_Querier_OOOQuery as that doesn't use copy, that is copyHeadChunk is false in the if condition above. Case 2: OOO mmaped head chunk overlaps with first mmaped in-order chunk. Query: |----------------------------------------------------------------| InO: |------mmap---------------||---------mem----------------------| OOO: |-----mmap-----------| |--mem--| In this case the meta contains the reference of the in-order chunk and no indication that a merge is needed with the OOO mmaped chunk. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-23 15:50:47 +02:00
Bryan Boreham	0538ad3a08	Merge pull request #14719 from aknuds1/arve/fix-valnone ... [COMMENT] tsdb: Fix ValNone typo	2024-08-23 09:32:59 +01:00
Arve Knudsen	b0aba26ed5	tsdb: Fix ValNone typo in comment ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-23 08:20:20 +02:00
Björn Rabenstein	5fd2717aec	Merge pull request #14715 from prometheus/beorn7/lint ... lint: Revamp our linting rules, mostly around doc comments	2024-08-22 17:56:40 +02:00
beorn7	0f760f63dd	lint: Revamp our linting rules, mostly around doc comments ... Several things done here: - Set `max-issues-per-linter` to 0 so that we actually see all linter warnings and not just 50 per linter. (As we also set `max-same-issues` to 0, I assume this was the intention from the beginning.) - Stop using the golangci-lint default excludes (by setting `exclude-use-default: false`. Those are too generous and don't match our style conventions. (I have re-added some of the excludes explicitly in this commit. See below.) - Re-add the `errcheck` exclusion we have used so far via the defaults. - Exclude the signature requirement `govet` has for `Seek` methods because we use non-standard `Seek` methods a lot. (But we keep other requirements, while the default excludes completely disabled the check for common method segnatures.) - Exclude warnings about missing doc comments on exported symbols. (We used to be pretty adamant about doc comments, but stopped that at some point in the past. By now, we have about 500 missing doc comments. We may consider reintroducing this check, but that's outside of the scope of this commit. The default excludes of golangci-lint essentially ignore doc comments completely.) - By stop using the default excludes, we now get warnings back on malformed doc comments. That's the most impactful change in this commit. It does not enforce doc comments (again), but _if_ there is a doc comment, it has to have the recommended form. (Most of the changes in this commit are fixing this form.) - Improve wording/spelling of some comments in .golangci.yml, and remove an outdated comment. - Leave `package-comments` inactive, but add a TODO asking if we should change that. - Add a new sub-linter `comment-spacings` (and fix corresponding comments), which avoids missing spaces after the leading `//`. Signed-off-by: beorn7 <beorn@grafana.com>	2024-08-22 17:36:11 +02:00
Bryan Boreham	95b3c49836	[TESTS] remote-write: fix slice init length (#14702) ... fix: fix slice init length	2024-08-22 12:53:45 +01:00
Björn Rabenstein	0a00d21cff	Merge pull request #14673 from machine424/docdoccc ... docs(configuration.md): clarify the explanations about some temp labe…	2024-08-22 12:31:46 +02:00
Arve Knudsen	a36d1a8a92	Merge pull request #14707 from aknuds1/arve/upgradegolangci-lint ... Upgrade golangci-lint to v1.60.2	2024-08-22 10:58:42 +02:00
Arve Knudsen	fbcd50f32c	Upgrade golangci-lint to v1.60.2 ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-21 18:57:17 +02:00
Arve Knudsen	7cfe0b1567	Merge pull request #14706 from aknuds1/arve/otlp-invalid-histograms ... OTLP receiver: Warn when encountering invalid exponential histograms	2024-08-21 18:07:45 +02:00
Arve Knudsen	b50c5d42fe	OTLP receiver: Warn when encountering invalid exponential histograms ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-21 16:43:16 +02:00
Bryan Boreham	9a74d53935	[BUGFIX] TSDB: Fix query overlapping in-order and ooo head (#14693) ... * tsdb: Unit test query overlapping in order and ooo head Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * TSDB: Merge overlapping head chunk The basic idea is that getOOOSeriesChunks can populate Meta.Chunk, but since it only returns one Meta per overlapping time-slot, that pointer may end up in a Meta with a head-chunk ID. So we need HeadAndOOOChunkReader.ChunkOrIterable() to call mergedChunks in that case. Previously, mergedChunks was checking that meta.Ref was a valid OOO chunk reference, but it never actually uses that reference; it just finds all chunks overlapping in time. So we can delete that code. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-21 14:24:20 +01:00
Björn Rabenstein	7fad1ec8ee	Merge pull request #14655 from suntala/suntala/sort-by-label-enhancement ... promql: Fall back to full label sets when sorting by label	2024-08-21 12:28:55 +02:00
cuishuang	2110661121	fix: fix slice init length ... Signed-off-by: cuishuang <imcusg@gmail.com>	2024-08-21 18:15:25 +08:00
suntala	8e82ac8d2b	Add comments to the sort by label functions ... Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-20 20:41:55 +02:00
Ben Kochie	1435c8ae4a	Include test CA text info (#14699) ... Use `openssl x509 -text -in <file>` to include the text version of the certificate in order to make it easier to see the diff when applying changes to the cert. Signed-off-by: SuperQ <superq@gmail.com>	2024-08-20 19:41:02 +02:00
Julien	f9f39a4411	Extend testing CA certificates (#14696) ... Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-08-20 17:09:07 +02:00
machine424	88cac6fb49	docs(configuration.md): clarify the explanations about some temp labels and format for better visibility. ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-08-20 15:44:39 +02:00
Julien	481d718539	Merge pull request #14661 from prymitive/TestHangingNotifier ... tests: increase TestHangingNotifier timeout	2024-08-20 14:37:16 +02:00
Julien Pivotto	9b5e7623f4	Add support for multiple listening addresses ... Fixes #9105 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-08-20 13:09:16 +02:00
Julien	82c4599ebe	Merge pull request #14690 from harshitasao/fix-vulnerability ... fix: fixed the vulnerability	2024-08-20 13:06:51 +02:00
Arve Knudsen	b914a9b580	Merge pull request #14691 from aknuds1/arve/upgrade-golangci-lint ... Upgrade golangci-lint to v1.60.1	2024-08-19 13:51:29 +02:00
Arve Knudsen	250aa5031d	Remove empty line ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-19 10:50:27 +02:00
Arve Knudsen	ee47150646	Update web/web.go ... Co-authored-by: Julien <291750+roidelapluie@users.noreply.github.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-19 10:43:17 +02:00
Arve Knudsen	4023c2405a	Update web/web.go ... Co-authored-by: Julien <291750+roidelapluie@users.noreply.github.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-19 10:43:05 +02:00
Arve Knudsen	3a78e76282	Upgrade golangci-lint to v1.60.1 ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-18 12:13:25 +02:00
harshitasao	3b0a386c62	fix: fixed the vulnerability ... Signed-off-by: harshitasao <harshitasao@gmail.com>	2024-08-18 13:15:28 +05:30
Ganesh Vernekar	e86e4ed87f	Merge pull request #14490 from ywwg/owilliams/utf8-02-upstream ... feat(utf8): utf-8 content negotiation and flags	2024-08-16 17:23:42 -04:00
Owen Williams	9e7308de38	feat(utf8): utf8 content negotation and flags ... Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-16 16:41:27 -04:00
Bryan Boreham	87dccb1d1b	Merge pull request #14649 from machine424/ftest ... fix(tsdb/db_test.go): close the corrupted chunk after creating it to satisfy Windows FS	2024-08-16 11:57:54 +01:00
Arve Knudsen	66388f706a	Merge pull request #14042 from aknuds1/arve/wlog-histograms ... tsdb/wlog: Only treat unknown record types as failure	2024-08-16 12:00:49 +02:00
Björn Rabenstein	1daf7cdd62	Merge pull request #14626 from cuiweiyuan/main ... chore: fix some function names	2024-08-15 11:46:21 +02:00
cuiweiyuan	1800af54f0	chore: fix some function names ... Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>	2024-08-15 13:57:21 +08:00
Arve Knudsen	b5d13a1ab5	Merge remote-tracking branch 'prometheus/main' into arve/wlog-histograms ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-14 19:04:53 +01:00
Bryan Boreham	d1ea6eb35d	Merge pull request #14354 from bboreham/ooo-head-together ... [PERF] TSDB: Query head and ooo-head together The current implementation of out-of-order querying runs two queriers which each iterate all series in the head, then runs a merge operation on the output. This PR adds HeadAndOOOQuerier which iterates just once over series, then where necessary merges chunks from in-order and out-of-order lists. In order to distinguish in-order from out-of-order chunk references I set bit 23 (i.e. 1<<23) on ooo references; this reduces the maximum number of chunks from 16 million to 8 million. Note one side-effect of this change is that results may come in a different order - the merge operation done previously required a sort of all series. This only changes where Prometheus does not guarantee the order. Fixes #11628	2024-08-14 14:39:30 +01:00
Bryan Boreham	512c67ec26	TSDB: Never go over maximum number of OOO chunks ... In `mmapCurrentOOOHeadChunk`, check if the number is at the maximum and drop the data with an error log. This is not expected to happen as the maximum is over 8 million; that's 8 years of 1 sample every second. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:59 +01:00
Bryan Boreham	9135da1e4f	TSDB: Review feedback ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Re-enable check in `createHeadWithOOOSamples` which wasn't really broken. * Move code making `Block` into a `Queryable` into test file. * Make `getSeriesChunks` return a slice (renamed `appendSeriesChunks`). * Rename `oooMergedChunks` to `mergedChunks`. * Improve comment on `ChunkOrIterableWithCopy`. * Name return values from unpackHeadChunkRef. Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com> Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:44 +01:00
Bryan Boreham	7ffd3ca280	TSDB: Cosmetic: move HeadAndOOO implementations where old code was ... This makes the diffs easier to follow. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	e95607b276	TSDB: Lock round access to labels, where necessary ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	26b3de0438	TSDB: Remove OOOHeadIndexReader ... Use headIndexReader instead. OOOCompactionHeadIndexReader needs to be expanded slightly, because it previously delegated to OOOHeadIndexReader. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	a299c7b6d6	TSDB: Remove OOOHeadChunkReader ... Use HeadAndOOOChunkReader instead. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	e7e50a3afd	TSDB: Remove code for querying OOO-head only ... Just query via `HeadAndOOOQuerier`, which will skip series where no in-order chunks are in range. Now we don't need `OOORangeHead`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	0a2ff76881	TSDB tests: Fix up BenchmarkQueries ... Was not working even on main. Some cases still error. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:04 +01:00