13605 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Bryan Boreham	26977b5c1e	Merge pull request #14624 from prometheus/backport-pr14538-to-2.54 ... Use CopyTo when resetting histogram in stats iterator	2024-08-14 15:19:46 +01:00
Bryan Boreham	03d5c634ac	Merge pull request #14623 from prometheus/backport-pr14514-to-2.54 ... Ignore stale histograms for counter reset detection	2024-08-14 15:02:35 +01:00
Bryan Boreham	d1ea6eb35d	Merge pull request #14354 from bboreham/ooo-head-together ... [PERF] TSDB: Query head and ooo-head together The current implementation of out-of-order querying runs two queriers which each iterate all series in the head, then runs a merge operation on the output. This PR adds HeadAndOOOQuerier which iterates just once over series, then where necessary merges chunks from in-order and out-of-order lists. In order to distinguish in-order from out-of-order chunk references I set bit 23 (i.e. 1<<23) on ooo references; this reduces the maximum number of chunks from 16 million to 8 million. Note one side-effect of this change is that results may come in a different order - the merge operation done previously required a sort of all series. This only changes where Prometheus does not guarantee the order. Fixes #11628	2024-08-14 14:39:30 +01:00
Bryan Boreham	512c67ec26	TSDB: Never go over maximum number of OOO chunks ... In `mmapCurrentOOOHeadChunk`, check if the number is at the maximum and drop the data with an error log. This is not expected to happen as the maximum is over 8 million; that's 8 years of 1 sample every second. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:59 +01:00
Bryan Boreham	9135da1e4f	TSDB: Review feedback ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Re-enable check in `createHeadWithOOOSamples` which wasn't really broken. * Move code making `Block` into a `Queryable` into test file. * Make `getSeriesChunks` return a slice (renamed `appendSeriesChunks`). * Rename `oooMergedChunks` to `mergedChunks`. * Improve comment on `ChunkOrIterableWithCopy`. * Name return values from unpackHeadChunkRef. Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com> Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:44 +01:00
Bryan Boreham	7ffd3ca280	TSDB: Cosmetic: move HeadAndOOO implementations where old code was ... This makes the diffs easier to follow. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	e95607b276	TSDB: Lock round access to labels, where necessary ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	26b3de0438	TSDB: Remove OOOHeadIndexReader ... Use headIndexReader instead. OOOCompactionHeadIndexReader needs to be expanded slightly, because it previously delegated to OOOHeadIndexReader. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	a299c7b6d6	TSDB: Remove OOOHeadChunkReader ... Use HeadAndOOOChunkReader instead. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	e7e50a3afd	TSDB: Remove code for querying OOO-head only ... Just query via `HeadAndOOOQuerier`, which will skip series where no in-order chunks are in range. Now we don't need `OOORangeHead`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:13 +01:00
Bryan Boreham	0a2ff76881	TSDB tests: Fix up BenchmarkQueries ... Was not working even on main. Some cases still error. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 13:41:04 +01:00
Björn Rabenstein	9849418fac	Merge pull request #14611 from charleskorn/sum-and-avg-over-mixed-custom-exponential-histograms ... promql: fix incorrect results and panics in `sum` and `avg` over mixed custom and exponential buckets, or incompatible custom buckets	2024-08-14 13:56:47 +02:00
suntala	ce4eac859a	Link to specific feature flag entry ... Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-14 13:22:12 +02:00
Bryan Boreham	f261597944	TSDB: Fix up LabelValues to work for OOO-only head ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 11:19:02 +01:00
Bryan Boreham	6529d6336c	TSDB: NewHeadAndOOOChunkReader takes headChunkReader ... So we can pass nil and have it read just OOO chunks. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 11:19:02 +01:00
Bryan Boreham	e04d137649	[PERF] TSDB: Query head and ooo-head together ... Add `HeadAndOOOQuerier` which iterates just once over series, then where necessary merges chunks from in-order and out-of-order lists. Add a ChunkQuerier for in-order and ooo together Add copy-last-chunk behaviour to HeadAndOOOChunkReader Out-of-order chunk IDs are distinguished from in-order by setting bit 23. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 11:19:02 +01:00
Bryan Boreham	2936ab80d7	[Tests] Promtool: Sort output where Prometheus does not guarantee the order. ... Previously this was working because iout-of-order chunks forced a sort and merge. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 11:19:02 +01:00
Bryan Boreham	da31da3ea6	Refactor: extract selectSeriesSet and selectChunkSeriesSet ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 11:19:02 +01:00
Bryan Boreham	7e24844d08	Refactor: extract headChunkReader.chunkFromSeries() ... For when you have a series locked already. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 11:19:02 +01:00
Bryan Boreham	a32aca0cd7	Refactoring: extract getOOOSeriesChunks ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 11:19:02 +01:00
Bryan Boreham	c75c8f8329	Refactoring: extract getSeriesChunks ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 11:19:02 +01:00
Bryan Boreham	0c852680bf	[Benchmark] TSDB: Add BenchmarkQuerierSelectWithOutOfOrder ... Refactor existing BenchmarkQuerierSelect to provide the set-up. Note that Head queries now run faster because they use a RangeHead. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-14 11:19:02 +01:00
George Krajcsovits	ff0a1e5e11	Merge pull request #14671 from krajorama/prep-ooo-native-histograms ... tsdb: prepare inserting native histograms into OOO head	2024-08-14 12:02:37 +02:00
György Krajcsovits	41656162fc	tsdb: prepare inserting native histograms into OOO head ... Rename a variable. Add parameters to memSeries.insert function. No effect on how float samples are handled. Related to #14546 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-14 11:13:47 +02:00
Charles Korn	52818a97e2	Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms ... # Conflicts: # promql/promqltest/testdata/native_histograms.test	2024-08-14 07:52:08 +10:00
Björn Rabenstein	c2bc6cfe97	Merge pull request #14621 from charleskorn/panic-message ... promql: clarify error message logged when panic occurs during query evaluation	2024-08-13 23:02:43 +02:00
George Krajcsovits	fe5354df88	Merge pull request #14658 from krajorama/zero-rate-nh-quantile ... Native histograms: define behavior when rate is null.	2024-08-13 16:05:43 +02:00
György Krajcsovits	386fc8b9f6	Update from review comments. ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-13 15:26:07 +02:00
Bryan Boreham	aa4b056ad0	Merge pull request #13200 from bboreham/wlog-defer ... tsdb/wlog: close segment files sooner	2024-08-13 14:11:38 +01:00
Arve Knudsen	6f1fd4be96	Merge pull request #14663 from aknuds1/arve/promql-comment ... PromQL engine: Fix comment regarding non-nil histogram pointer	2024-08-13 09:11:40 +02:00
Arve Knudsen	0503d4f372	PromQL: Fix comment regarding non-nil histogram pointer ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-13 08:55:24 +02:00
Suraj Nath	74c1af3fd5	address review comments ... Signed-off-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>	2024-08-12 23:36:48 +05:30
Lukasz Mierzwa	7694c89497	Increase TestHangingNotifier timeout ... This test keeps timing out on our arm64 CI server, it does use a very slow timeout and that 5ms doesn't seem to be enough. But it 10x. Signed-off-by: Lukasz Mierzwa <lukasz@cloudflare.com>	2024-08-12 14:53:08 +01:00
György Krajcsovits	6aee5b4b38	fix typo ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-12 12:04:45 +02:00
György Krajcsovits	06a8886b94	Native histograms: define behavior when rate is null. ... Histogram quantile returns NaN in this case, which might be surprising, so add a unit test that clarifies that this is intentional. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-12 10:40:21 +02:00
suntala	532904a1d6	Document changes to sort by label ... Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com> Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-11 21:44:03 +02:00
suntala	fd2f44af7f	Fall back to comparing by label set when sorting by label desc ... Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com> Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-11 21:44:03 +02:00
suntala	94ad489328	Fall back to comparing by label set when sorting by label ... Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com> Signed-off-by: suntala <arati.rana@grafana.com>	2024-08-11 21:44:03 +02:00
Suraj Nath	a3b36c1225	pin to SHA of github action ... Signed-off-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>	2024-08-09 23:06:11 +05:30
Suraj Nath	5dcaaee72a	add stale check in list of files to sync ... Signed-off-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>	2024-08-09 23:01:16 +05:30
Suraj Nath	7f0254da29	chore: migrate stale check to use stale github action ... Signed-off-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>	2024-08-09 23:01:16 +05:30
Arve Knudsen	5fd66ba855	Merge pull request #14510 from aknuds1/arve/add-normalize-name-tests ... otlptranslator: Add tests for BuildCompliantName	2024-08-09 18:21:16 +02:00
machine424	82f38d3e9a	fix(tsdb/db_test.go): close the corrupted chunk after creating it to satisfy Windows FS ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-08-09 14:53:57 +02:00
Bryan Boreham	5354e87a70	Merge pull request #14644 from prometheus/prep-2.54.0 ... Cut release 2.54.0	2024-08-09 12:34:14 +01:00
Bryan Boreham	8978f3ef71	Cut release 2.54.0 ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-09 12:07:28 +01:00
Julien	616038f2b6	Merge pull request #14553 from prometheus/dependabot/go_modules/github.com/aws/aws-sdk-go-1.55.5 ... build(deps): bump github.com/aws/aws-sdk-go from 1.54.19 to 1.55.5	2024-08-09 12:06:36 +02:00
Julien	7b0f6f1aed	Merge pull request #14573 from prometheus/dependabot/github_actions/scripts/actions/setup-go-5.0.2 ... build(deps): bump actions/setup-go from 5.0.1 to 5.0.2 in /scripts	2024-08-09 12:06:05 +02:00
Julien	c210917b91	Merge pull request #14571 from prometheus/dependabot/github_actions/actions/upload-artifact-4.3.4 ... build(deps): bump actions/upload-artifact from 4.3.3 to 4.3.4	2024-08-09 12:05:37 +02:00
Julien	4387de8189	Merge pull request #14568 from prometheus/dependabot/github_actions/bufbuild/buf-setup-action-1.35.1 ... build(deps): bump bufbuild/buf-setup-action from 1.34.0 to 1.35.1	2024-08-09 12:05:24 +02:00
Charles Korn	f992f81bd0	Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms ... Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com>	2024-08-09 13:58:54 +10:00