224 commits

Author	SHA1	Message	Date
Bryan Boreham	7b03796d0f	Scraping: stop storing discovered labels (#15261) ... Instead of storing discovered labels on every target, recompute them if required. The `Target` struct now needs to hold some more data required to recompute them, such as ScrapeConfig. This moves the load from every Prometheus all of the time, to just when someone views Service Discovery in the UI. The way `PopulateLabels` is used changes; you are no longer expected to call it with a part-populated `labels.Builder`. The signature of `Target.Labels` changes to take a `labels.Builder` instead of a `ScratchBuilder`, for consistency with `DiscoveredLabels`. This will save a lot of work when many targets are filtered out in relabeling. Combine with `keep_dropped_targets` to avoid ever computing most labels for dropped targets. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-12-21 13:33:08 +00:00
Bryan Boreham	c69aeabf1d	Merge pull request #15563 from GiedriusS/fix_setting_field ... scrape: fix nil panic after scrape loop reload	2024-12-10 15:33:56 +00:00
Giedrius Statkevičius	5d76510bd6	scrape: update test name ... Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-12-09 17:36:36 +02:00
Giedrius Statkevičius	5479fb3cfd	scrape: update test after review ... Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-12-09 08:18:06 +02:00
Bryan Boreham	48287b15d4	Merge pull request #13914 from bboreham/scrape-reload-metric ... [REFACTOR] Scraping: rename variables for clarity	2024-12-02 17:09:25 +00:00
Giedrius Statkevičius	90e832c861	scrape: fix nil panic after scrape loop reload ... Don't forget to set `metrics` field as otherwise scraping will lead to a nil panic in case the body size limit is reached. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-12-02 16:08:37 +02:00
Arve Knudsen	89bbb885e5	Upgrade to golangci-lint v1.62.0 (#15424) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-11-20 17:22:20 +01:00
Bryan Boreham	26886d6d95	Merge branch 'release-3.0' into merge-release-into-main	2024-11-18 14:43:56 +00:00
Bryan Boreham	3f6a133f26	[Test] Scraping: check reload metrics ... Need to extend `newTestScrapeMetrics`` to get at the Registry. `gatherLabels` function could go upstream to `client_golang`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-11-07 17:29:04 +00:00
alexgreenbank	6b09f094e1	scrape: stop erroring on empty scrapes ... Signed-off-by: alexgreenbank <alex.greenbank@grafana.com>	2024-11-07 11:30:03 +00:00
Matthieu MOREL	af1a19fc78	enable errorf rule from perfsprint linter ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2024-11-06 16:50:36 +01:00
Vanshika	cccbe72514	TSDB: Fix some edge cases when OOO is enabled (#14710) ... Fix some edge cases when OOO is enabled Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> Signed-off-by: Vanshika <102902652+Vanshikav123@users.noreply.github.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>	2024-10-23 17:34:28 +02:00
George Krajcsovits	aa81210c8b	NHCB scrape: refactor state handling and speed up scrape test (#15193) ... * NHCB: scrape use state field and not booleans From comment https://github.com/prometheus/prometheus/pull/14978#discussion_r1800898724 Also make compareLabels read only and move storeLabels to the first processed classic histogram series. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Speed up TestConvertClassicHistogramsToNHCB 3x Reduce the startup time and timeouts Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * lint fix Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-22 17:49:25 +01:00
György Krajcsovits	4283ae73dc	Rename convert_classic_histograms to convert_classic_histograms_to_nhcb ... On reviewer request. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-21 13:22:58 +02:00
György Krajcsovits	a23aed5634	More followup to #15164 ... Scrape test for NHCB modified. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-21 11:10:50 +02:00
György Krajcsovits	70742a64aa	Follow up #15178 ... Renaming Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-21 11:03:47 +02:00
György Krajcsovits	8c1b5a6251	Merge branch 'main' into nhcb-scrape-impl	2024-10-21 11:00:41 +02:00
Alex Greenbank	421a3c22ea	scrape: provide a fallback format (#15136) ... scrape: Remove implicit fallback to the Prometheus text format Remove implicit fallback to the Prometheus text format in case of invalid/missing Content-Type and fail the scrape instead. Add ability to specify a `fallback_scrape_protocol` in the scrape config. --------- Signed-off-by: alexgreenbank <alex.greenbank@grafana.com> Signed-off-by: Alex Greenbank <alex.greenbank@grafana.com> Co-authored-by: Björn Rabenstein <beorn@grafana.com>	2024-10-18 17:12:31 +02:00
Bartlomiej Plotka	efc43d0714	s/scrape_classic_histograms/always_scrape_classic_histograms (3.0 breaking change) (#15178) ... This is for readability, especially when we can converting to nhcb option. See discussion https://cloud-native.slack.com/archives/C077Z4V13AM/p1729155873397889 Signed-off-by: bwplotka <bwplotka@gmail.com>	2024-10-18 08:32:15 +01:00
György Krajcsovits	04b827dd77	Merge branch 'main' into nhcb-scrape-impl	2024-10-17 12:34:44 +02:00
Yi	2cabd1b707	config: remove expand-external-labels flag in release 3.0 (#14657) ... remove expand-external-labels feature flag and enabled env arg expansion for external labels by default. Signed-off-by: jyz0309 <45495947@qq.com>	2024-10-17 10:25:05 +02:00
György Krajcsovits	a2805e7934	Merge branch 'main' into nhcb-scrape-impl	2024-10-09 11:53:26 +02:00
TJ Hoplock	6ebfbd2d54	chore!: adopt log/slog, remove go-kit/log ... For: #14355 This commit updates Prometheus to adopt stdlib's log/slog package in favor of go-kit/log. As part of converting to use slog, several other related changes are required to get prometheus working, including: - removed unused logging util func `RateLimit()` - forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger - move some of the json file logging functionality to use prom/common package functionality - refactored some of the new json file logging for scraping - changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers - updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition - added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2024-10-07 15:58:50 -04:00
György Krajcsovits	ed2e7dc258	Merge branch 'main' into nhcb-scrape-impl ... # Conflicts: # scrape/scrape.go	2024-10-07 11:23:44 +02:00
Matthieu MOREL	ab64966e9d	fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094) ... * fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" --------- Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-10-06 16:35:29 +00:00
Manik Rana	f1c57a95ed	change: No longer ingest OM _created as timeseries if feature-flag 'enable-ct-zero-ingestion' is enabled; fixed OM text CT conversion bug (#14738) ... * chore: revert TypeRequiresCT to private Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: init NewOpenMetricsParser with skipCT true Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: allow opt-in to OM CT ingestion Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: use textparse interface to set om options Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: set skipOMSeries in test Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * wip: add tests for OM CR parse Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: merge ct tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add cases for OM text Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: check correct test cases Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: use both scrape protocols in config Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: fix inputs and output tests for OM Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: cleanup Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: rename skipOMSeries to skipOMCTSeries Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * fix: finish refac Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: move setup code outside test Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: verify _created lines create new metric in certain cases Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: post merge fixes Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * manager: Fixed CT OMText conversion bug; Refactored tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: imports Signed-off-by: Manik Rana <manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	2024-10-02 11:52:03 +01:00
George Krajcsovits	79a6238e19	scrape/scrape_test.go: reduce the time it takes to reload the manager (#14447) ... * scrape/scrape_test.go: reduce the time it takes to reload the manager TestNativeHistogramMaxSchemaSet took over 3x5s to complete because there's a minimum reload interval. I've made the testcases run in parallel and reduced the reload interval to 10ms. Now the test runs in around 0.1-0.2 seconds. Ran test 10000 times to check if it's flaky. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-09-26 18:35:15 +02:00
Arthur Silva Sens	6bd9b1a7cc	Histogram CT Zero ingestion ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-09-26 11:29:22 -03:00
György Krajcsovits	71fd2d93a9	Merge branch 'main' into nhcb-scrape-impl ... # Conflicts: # config/config.go # scrape/scrape.go	2024-09-25 13:43:57 +02:00
Jeanette Tan	90c266845b	fix lint ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:38:27 +02:00
Jeanette Tan	050b5fc257	refine test cases according to spec ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:38:22 +02:00
Jeanette Tan	de9de320a4	start to cover all test cases for scrape ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:38:18 +02:00
Jeanette Tan	f35c6649e4	don't blindly convert series with the classic histogram name suffixes if they are not actually histograms based on metadata ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:38:15 +02:00
Jeanette Tan	8b3ae15ad5	expand tests for classic and exponential native histograms ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:38:10 +02:00
Jeanette Tan	e3899187da	expand tests for protobuf and fix problems ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:38:08 +02:00
Jeanette Tan	cd498964e6	expand tests and support conversion to nhcb in the middle of scrape ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:38:04 +02:00
Jeanette Tan	f596f17024	allow option to convert classic histograms to nhcb entirely (don't append classic histogram series) ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:37:30 +02:00
Jeanette Tan	0a321fe4d8	improve new scrape test ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:37:20 +02:00
Jeanette Tan	62e7f0438d	implement basic conversion of classic to nhcb in scrape ... Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	2024-09-25 13:36:56 +02:00
Jan Fajerski	91608c002f	Merge branch 'main' into release-3.0-beta.0 ... Conflicts: scrape/scrape_test.go Pick both changes.	2024-09-10 20:51:20 +02:00
Björn Rabenstein	482f1f608c	Merge pull request #14880 from prometheus/njpm/fix-utf8-config ... fix(utf8): propagate validationScheme config to scraping options	2024-09-10 14:17:44 +02:00
Nicolás Pazos	569b6abfa3	fix(utf8): propagate validationScheme config to scraping options ... Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>	2024-09-09 14:09:49 -03:00
Jan Fajerski	fa318711f4	Merge branch 'main' into 3.0-main-sync-24-09-09 ... Conflicts: cmd/prometheus/main.go docs/command-line/prometheus.md docs/feature_flags.md web/ui/build_ui.sh web/web.go Resolved by dropping the UTF-8 feature flag and adding the `auto-reload-config` feature flag. For the new web ui pick all changes from `main`.	2024-09-09 15:44:22 +02:00
Julien	0a88943594	Scrape: test for q-value compliance with RFC 9110 in Accept header ... Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-09-09 12:52:00 +02:00
Björn Rabenstein	694d98032b	Merge pull request #14705 from prometheus/owilliams/default-on ... utf8: enable utf-8 support by default	2024-09-06 17:24:37 +02:00
Owen Williams	88bb05c3e8	utf8: enable utf-8 support by default ... This change causes Prometheus to allow all UTF-8 characters in metric and label names. This means that names that were previously invalid and would have been previously rejected will be allowed through. Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-09-06 08:48:11 -04:00
Julien Pivotto	bd9129117e	Target parameter labels should not be overridden by config params ... The following configuration snippet calls http://127.0.0.1:8000/?foo=value1 instead of http://127.0.0.1:8000/?foo=value2. This is incorrect, the labels of the target should be prefered. ```yaml - job_name: local params: foo: [value1] static_configs: - targets: ['127.0.0.1:8000'] labels: __param_foo: value2 ``` Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-09-05 12:10:11 +02:00
Julien	ce0f09b125	Scrape: Add scrape_failure_log_file to log Scrape Failures ... Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-09-05 11:01:40 +02:00
Owen Williams	9da75328ea	fix(utf8): ensure correct validation when legacy mode turned on (#14736) ... fix(utf8): ensure correct validation when legacy mode turned on This depends on the included update of the prometheus/common dependency. --------- Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-28 17:15:42 +02:00
Bryan Boreham	4202be5e79	Merge branch 'release-2.54' into merge-2.54.1-into-main	2024-08-27 12:04:48 +01:00