870 commits

Author	SHA1	Message	Date
Bryan Boreham	7b03796d0f	Scraping: stop storing discovered labels (#15261) ... Instead of storing discovered labels on every target, recompute them if required. The `Target` struct now needs to hold some more data required to recompute them, such as ScrapeConfig. This moves the load from every Prometheus all of the time, to just when someone views Service Discovery in the UI. The way `PopulateLabels` is used changes; you are no longer expected to call it with a part-populated `labels.Builder`. The signature of `Target.Labels` changes to take a `labels.Builder` instead of a `ScratchBuilder`, for consistency with `DiscoveredLabels`. This will save a lot of work when many targets are filtered out in relabeling. Combine with `keep_dropped_targets` to avoid ever computing most labels for dropped targets. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-12-21 13:33:08 +00:00
bwplotka	f44bba31b3	config: Remove validation GetScrapeConfigs; require using config.Load. ... We should never modify (or even shallow copy) Config after config.Load; added comments and modified GetScrapeConfigs to do so. For GetScrapeConfigs the validation (even repeated) was likely doing writes (because global fields was 0). We GetScrapeConfigs concurrently in tests and ApplyConfig causing test races. In prod there were races but likelyt only to replace 0 with 0, so not too severe. I removed validation since I don't see anyone using our config.Config without Load. I had to refactor one test that was doing it, all others use yaml config. Fixes #15538 Previous attempt: https://github.com/prometheus/prometheus/pull/15634 Signed-off-by: bwplotka <bwplotka@gmail.com>	2024-12-11 13:31:01 +00:00
Bryan Boreham	dd1d707e1c	Merge pull request #15371 from bboreham/main-naming ... [REFACTOR] Small improvement to top-level naming	2024-11-24 17:55:10 +00:00
Ben Ye	40c3ba9fd4	fix promtool analyze block shows metric name with 0 cardinality (#15438) ... Signed-off-by: Ben Ye <benye@amazon.com>	2024-11-24 17:30:20 +01:00
Bryan Boreham	3c22b35a03	Merge pull request #15339 from colega/export-go-sync-mutex-wait-total-seconds-total-metric ... Export 'go_sync_mutex_wait_total_seconds_total' metric	2024-11-20 13:34:37 +00:00
Björn Rabenstein	384c5951ef	Merge pull request #14489 from harry671003/implement_metadata_limit ... storage: Implement limit in mergeGenericQuerier	2024-11-19 17:32:16 +01:00
Ben Kochie	869addfec8	Enable auto-gomaxprocs by default (#15378) ... Enable the `auto-gomaxprocs` feature flag by default. * Add command line flag `--no-auto-gomaxprocs` to disable. Signed-off-by: SuperQ <superq@gmail.com>	2024-11-12 13:27:01 +01:00
Bryan Boreham	f0bd5fdcc6	Merge pull request #15226 from aniketnk/i15185 ... Parallelize tests in cmd/promtool/	2024-11-12 11:40:30 +00:00
Julien	e9179cb240	Merge pull request #15011 from roidelapluie/fixreload ... Fix auto reload when a config file with a syntax error is reverted	2024-11-12 11:55:21 +01:00
Ben Kochie	0f0deb77a2	Enable auto-gomemlimit by default (#15372) ... Enable the `auto-gomemlimit` feature flag by default. * Add command line flag `--no-auto-gomemlimit` to disable. Signed-off-by: SuperQ <superq@gmail.com>	2024-11-11 18:26:38 +01:00
Bryan Boreham	de35a40ed5	[REFACTOR] Small improvement to top-level naming ... Previously, top-level initialization (including WAL reading) had this at the top of the stack: github.com/oklog/run.(*Group).Run.func1:38 I found this confusing, and thought it had something to do with logging. Introducing another local function should make this clearer. Also make the error message user-centric not code-centric. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-11-11 10:55:05 +00:00
Ben Ye	140f4aa9ae	feat: Allow customizing TSDB postings decoder (#13567) ... * allow customizing TSDB postings decoder --------- Signed-off-by: Ben Ye <benye@amazon.com>	2024-11-11 07:59:24 +01:00
Aniket Kaulavkar	650abaccce	Parallelize tests in cmd/promtool/ ... Signed-off-by: Aniket Kaulavkar <aniket.kaulavkar@gmail.com>	2024-11-08 21:54:08 +05:30
🌲 Harry 🌊 John 🏔	f9bc50b247	storage: Implement limit in mergeGenericQuerier ... Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>	2024-11-07 09:08:23 -08:00
Arthur Silva Sens	96c2fe04e7	Merge pull request #15328 from prometheus/deadcode ... chore: Remove dead code	2024-11-06 13:20:53 -03:00
Matthieu MOREL	af1a19fc78	enable errorf rule from perfsprint linter ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2024-11-06 16:50:36 +01:00
Oleg Zaytsev	50e83e8f98	Export 'go_sync_mutex_wait_total_seconds_total' metric ... This metric can be useful to debug mutex contention. Ref: https://github.com/golang/go/issues/49881 Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	2024-11-05 15:25:55 +01:00
Julien	9d9e86b333	Fix auto reload when a config file with a syntax error is reverted ... When we had a syntax error but restored the old file, we did not re-trigger the config reload, so the config reload metric was showing that config reload was unsucessful. I made magic to handle logs in cmd/prometheus. For now it is a separate file so we can backport this easily. I will generalize the helper in another PR. Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-11-05 12:43:34 +01:00
Arthur Silva Sens	88818c9cb3	chore: Remove dead code ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-11-04 13:00:22 -03:00
Alban Hurtaud	4b56af7eb8	Add hidden flag for the delayed compaction random time window (#14919) ... * Add hidden flag for the delayed compaction random time window Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Update cmd/prometheus/main.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> * Update cmd/prometheus/main.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> * Update tsdb/db.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> * Fix flag name according to review - add test for delay Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Fix afer main rebase Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Implement review comments Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Update generatedelaytest to try with limit values Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> --------- Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>	2024-11-04 08:26:26 +01:00
Vanshika	cccbe72514	TSDB: Fix some edge cases when OOO is enabled (#14710) ... Fix some edge cases when OOO is enabled Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> Signed-off-by: Vanshika <102902652+Vanshikav123@users.noreply.github.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>	2024-10-23 17:34:28 +02:00
George Krajcsovits	1b4e7f74e6	feat(tools): add debug printouts to rules unit testing (#15196) ... * promtool: Add debug flag for rule tests This makes it print out the tsdb state (both input_series and rules that are run) at the end of a test, making reasoning about tests much easier. Signed-off-by: David Leadbeater <dgl@dgl.cx> * Reuse generated test name from junit testing Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: David Leadbeater <dgl@dgl.cx> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: David Leadbeater <dgl@dgl.cx>	2024-10-22 15:24:36 +02:00
Bryan Boreham	70e2d23027	Merge pull request #11474 from clwluvw/group-label ... [FEATURE] rules: add labels at group level	2024-10-21 14:47:12 +01:00
machine424	cf128a0472	test(cmd/prometheus): speed up test execution by t.Parallel() when possible ... turn some loops into subtests to make use of t.Parallel() requires Go 1.22 to make use of https://go.dev/blog/loopvar-preview Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-10-18 17:52:59 +02:00
Alan Protasio	c78d5b94af	Disallowing configure AM with the v1 api (#13883) ... * Stop supporting Alertmanager v1 * Disallowing configure AM with the v1 api Signed-off-by: alanprot <alanprot@gmail.com> * Update config/config_test.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * Update config/config.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * Addressing coments Signed-off-by: alanprot <alanprot@gmail.com> * Update notifier/notifier.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> * Update config/config_test.go Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> --------- Signed-off-by: alanprot <alanprot@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com>	2024-10-18 15:23:14 +02:00
Yi	2cabd1b707	config: remove expand-external-labels flag in release 3.0 (#14657) ... remove expand-external-labels feature flag and enabled env arg expansion for external labels by default. Signed-off-by: jyz0309 <45495947@qq.com>	2024-10-17 10:25:05 +02:00
TJ Hoplock	6ebfbd2d54	chore!: adopt log/slog, remove go-kit/log ... For: #14355 This commit updates Prometheus to adopt stdlib's log/slog package in favor of go-kit/log. As part of converting to use slog, several other related changes are required to get prometheus working, including: - removed unused logging util func `RateLimit()` - forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger - move some of the json file logging functionality to use prom/common package functionality - refactored some of the new json file logging for scraping - changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers - updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition - added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2024-10-07 15:58:50 -04:00
Julien	9d8f93ea92	Merge pull request #15113 from yeya24/add-missing-allow-overlapping-compaction-flag ... Add missing flag storage.tsdb.allow-overlapping-compaction	2024-10-07 11:17:24 +02:00
Matthieu MOREL	ab64966e9d	fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094) ... * fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" --------- Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-10-06 16:35:29 +00:00
Ben Ye	6c3d11629b	add missing flag storage.tsdb.allow-overlapping-compaction ... Signed-off-by: Ben Ye <benye@amazon.com>	2024-10-05 17:05:42 -07:00
Julien	297667469a	Merge pull request #15090 from roidelapluie/querylogrule ... Fix flakiness of QueryLogTest	2024-10-04 14:57:24 +02:00
Julien	9d275c23de	cmd/prometheus: Fix flakiness of QueryLogTest ... Now we check that a rule execution has taken place. This also reduces the time to run the rules tests from 45s to 25s. Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-10-04 11:23:57 +02:00
Julien	21e0f83b68	Move notifications in utils ... Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-10-04 10:11:56 +02:00
Julien	b6158e8956	Notify web UI when starting up and shutting down ... Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-10-03 10:27:04 +02:00
Jesus Vazquez	77d3b3aff3	OTLP: Remove experimental word form OTLP receiver (#14894) ... The OTLP receiver can now considered stable. We've had it for longer than a year in main and has received constant improvements. Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>	2024-10-01 14:36:52 +02:00
Julien	f9bbad1148	Limit the number of SSE Subscribers to 16 by default ... Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-09-27 15:51:51 +02:00
Julien	7aa4721373	Merge pull request #14946 from roidelapluie/notifications ... Add notifications to the Web UI	2024-09-27 15:50:43 +02:00
Julien	6cde0096e2	Add notifications to the web UI when configuration reload fails. ... This commit introduces a new `/api/v1/notifications/live` endpoint that utilizes Server-Sent Events (SSE) to stream notifications to the web UI. This is used to display alerts such as when a configuration reload has failed. I opted for SSE over WebSockets because SSE is simpler to implement and more robust for our use case. Since we only need one-way communication from the server to the client, SSE fits perfectly without the overhead of establishing and maintaining a two-way WebSocket connection. When the SSE connection fails, we go back to a classic /api/v1/notifications API endpoint. This commit also contains the required UI changes for the new Mantine UI. Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-09-27 15:28:38 +02:00
Bryan Boreham	410fcce6f0	Remove unnecessary pprof import (#14988) ... The pattern of `import _ "net/http/pprof"` adds handlers to the default http handler, but Prometheus does not use that. There are explicit handlers in `web/web.go`. So, we can remove this line with no impact to behaviour. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-09-27 07:45:49 +01:00
Björn Rabenstein	f74722841b	Merge pull request #14160 from alex-kattathra-johnson/issue-13959 ... Remove no-default-scrape-port featureFlag	2024-09-26 18:45:56 +02:00
Arthur Silva Sens	6bd9b1a7cc	Histogram CT Zero ingestion ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-09-26 11:29:22 -03:00
Alex Johnson	be0f10054e	Remove no-default-scrape-port featureFlag ... Signed-off-by: Alex Johnson <alex.kattathra.johnson@gmail.com>	2024-09-25 10:13:19 -05:00
Julien	919648cafc	Merge pull request #14947 from roidelapluie/reloadinvalidyaml ... fix(autoreload): Reload invalid yaml files	2024-09-23 10:03:23 +02:00
Björn Rabenstein	5b9148e552	Merge pull request #14820 from charleskorn/promqltest-native-histogram-format ... promqltest: use test expression format for histograms in assertion failure messages and include reset hint in the test expression	2024-09-20 16:47:08 +02:00
Julien	146b22d196	fix(autoreload): Reload invalid yaml files ... When a YAML file is invalid, trigger auto-reload anyway so that user is aware that the configuration file is incorrect. Failing to do so does not change the reload status in metrics and api. Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-09-20 13:38:05 +02:00
Julius Volz	b8d1336d42	Merge pull request #14912 from roidelapluie/notready ... mantine UI: Distinguish between Not Ready and Stopping	2024-09-17 19:40:13 +02:00
Julien	ac5377873f	mantine UI: Distinguish between Not Ready and Stopping ... Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-09-17 16:02:16 +02:00
Carrie Edwards	14e3c05ce8	tsdb: Add support for ingestion of out-of-order native histogram samples (#14546) ... Add support for ingesting OOO native histograms * Add flag for enabling and disabling OOO native histogram ingestion * Update OOO querying tests to include native histogram samples * Add OOO head tests * Add test for OOO native histogram counter reset headers Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Jeanette Tan <jeanette.tan@grafana.com> Co-authored by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored by: Fiona Liao <fiona.liao@grafana.com>	2024-09-17 11:19:06 +02:00
Augustin Husson	6febfbb3be	put back feature flag 'delayed-compaction' and 'old-ui' (#14909) ... Signed-off-by: Augustin Husson <husson.augustin@gmail.com>	2024-09-16 15:46:28 +02:00
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868) ... * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	2024-09-10 22:32:03 +02:00