14391 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Julius Volz	51866b9fee	Merge pull request #15244 from prometheus/utf8-web-ui-support ... Support UTF-8 metric names and labels in web UI	2024-11-04 12:13:41 +01:00
Alban Hurtaud	4b56af7eb8	Add hidden flag for the delayed compaction random time window (#14919) ... * Add hidden flag for the delayed compaction random time window Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Update cmd/prometheus/main.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> * Update cmd/prometheus/main.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> * Update tsdb/db.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> * Fix flag name according to review - add test for delay Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Fix afer main rebase Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Implement review comments Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> * Update generatedelaytest to try with limit values Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> --------- Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com> Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>	2024-11-04 08:26:26 +01:00
Bryan Boreham	a1a78a151d	Merge pull request #15316 from bboreham/revert-15141 ... Revert "Fix `MemPostings.Add` and `MemPostings.Get` data race (#15141)"	2024-11-03 14:49:27 +00:00
Arve Knudsen	7c4f877881	otlptranslator: Harmonize non-UTF8 sanitization w/ naming rules. (#15314) ... * otlptranslator: Harmonize non-UTF8 sanitization w/ naming rules. Harmonize non-UTF8 sanitization w/ Prometheus naming rules. --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-11-03 12:56:54 +00:00
Bryan Boreham	2fbbfc3da8	Revert "Fix MemPostings.Add and MemPostings.Get data race (#15141)" ... This reverts commit 50ef0dc954. Memory allocation goes so high in Prombench that the system is unusable. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-11-03 12:30:34 +00:00
Bryan Boreham	9700933d18	Merge pull request #15311 from prometheus/remove-55-rc ... [BUILD] React-app: replace 0.55.0-rc.0 with 0.55.0	2024-11-03 12:30:02 +00:00
Arthur Silva Sens	1a22b1d846	bugfix: Fix otlp translator switching colons to undescores (#15251) ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-11-03 13:15:49 +01:00
Arthur Silva Sens	034d2b24bc	Fix typos in tests (#15312) ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-11-02 15:38:51 +01:00
Bryan Boreham	adcc873d51	[BUILD] React-app: replace 0.55.0-rc.0 with 0.55.0 ... Try to stop build errors. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-11-02 11:55:52 +00:00
Bryan Boreham	e2e01c1cff	Merge pull request #15216 from yeya24/log-last-series-labels ... log last series labelset when hitting OOO series labels	2024-11-01 14:15:39 +00:00
Bartlomiej Plotka	dd4eb4590d	Merge pull request #15222 from jmichalek132/jm-add-tcp-events-to-tracing ... tracing: add tcp events to remote store span	2024-11-01 14:26:15 +01:00
Arthur Silva Sens	ccaff318ff	Merge pull request #15249 from prometheus/otlp-fix-sanitiza ... bugfix: Fix otlp translator for foreign characters	2024-10-31 19:05:37 -03:00
Jan Fajerski	284135a905	Merge pull request #15223 from jan--f/release-3.0.0-rc.0 ... Prepare release 3.0.0 rc.0	2024-10-31 20:33:37 +01:00
Jan Fajerski	5fdd382ec3	CHANGELOG: remove duplicate entry ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-10-31 16:22:59 +01:00
Arthur Silva Sens	ea06f1a1d1	bugfix: Fix otlp translation of foreign characters ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-10-30 19:29:31 -03:00
Arthur Silva Sens	8588289c24	otlp translator: Add test showing bugs ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-10-30 19:28:53 -03:00
Julius Volz	76ca7d08d9	Fixup: re-add erroneously removed lines ... Signed-off-by: Julius Volz <julius.volz@gmail.com>	2024-10-30 16:43:10 +01:00
Juraj Michalek	76ff12b32a	chore: only create span events ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
Juraj Michalek	7ecdb55b02	chore: use otelhttptrace instead ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
Juraj Michalek	3c1ffbb2fd	chore: added idleTimeMs and fixed linting issues ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
Juraj Michalek	2e7c739d44	chore: add tcp events to remote store span ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
Julius Volz	c861b31b72	Support UTF-8 metric names and labels in web UI ... Fixes most of https://github.com/prometheus/prometheus/issues/15202 This should address all areas of the UI except for the autocompletion in the codemirror-promql text editor. The strategy here is that any time we print or internally serialize (like for the PromLens tree view) either a metric name or a label name as part of a selector or in other relevant parts of PromQL, we check whether it contains characters beyond what was previously supported, and if so, quote and escape it. In the case of metric names, we also have to move them from the beginning of the selector into the curly braces. Signed-off-by: Julius Volz <julius.volz@gmail.com>	2024-10-29 20:22:52 +01:00
Ganesh Vernekar	d0eecb1223	Merge pull request #15239 from colega/revert-14975-process-mempostings-delete-with-gomaxprocs-workers ... Revert "Process `MemPostings.Delete()` with `GOMAXPROCS` workers"	2024-10-29 12:27:43 -04:00
Oleg Zaytsev	ba11a55df4	Revert "Process MemPostings.Delete() with GOMAXPROCS workers" ... Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	2024-10-29 17:13:40 +01:00
Jan Fajerski	e2f55c34c9	fix CHANGELOG formatting and add entry for #14694 ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-10-29 14:41:50 +01:00
Nicolas Takashi	b6c538972c	[REFACTORY] simplify appender commit (#15112) ... * [REFACTOR] simplify appender commit Signed-off-by: Nicolas Takashi <nicolas.tcs@hotmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-10-29 12:34:02 +00:00
Ayoub Mrini	350e0d5bc1	Merge pull request #15228 from shenpengfeng/main ... chore: fix function name in comment	2024-10-29 12:12:59 +01:00
shenpengfeng	a44db5f784	chore: fix function name in comment ... Signed-off-by: shenpengfeng <xinhangzhou@icloud.com>	2024-10-29 17:58:44 +08:00
Arve Knudsen	706dcfeecf	tsdb.CircularExemplarStorage: Avoid racing (#15231) ... * tsdb.CircularExemplarStorage: Avoid racing --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-10-29 10:40:46 +01:00
Ayoub Mrini	559722dc68	Merge pull request #15206 from multani/fix-doc-formatting ... doc: fix formatting	2024-10-28 22:53:55 +01:00
Ayoub Mrini	99c0e68013	Merge pull request #15230 from GiedriusS/optimize_resolvepodref ... discovery/kubernetes: optimize resolvePodRef	2024-10-28 22:53:13 +01:00
Jan Fajerski	5408184452	react-app: bump version in lock file ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-10-28 17:10:34 +01:00
Jack Westbrook	7cda23ba32	fix(lezer-promql): fix missing types export in package.json (#15161) ... Signed-off-by: Jack Westbrook <jack.westbrook@gmail.com>	2024-10-28 17:05:10 +01:00
Giedrius Statkevičius	58fedb6b61	discovery/kubernetes: optimize more gets ... Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-10-28 17:17:37 +02:00
Giedrius Statkevičius	716fd5b11f	discovery/kubernetes: use namespacedName ... Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-10-28 16:19:56 +02:00
Giedrius Statkevičius	e452308e37	discovery/kubernetes: optimize resolvePodRef ... resolvePodRef is in a hot path: ``` ROUTINE ======================== github.com/prometheus/prometheus/discovery/kubernetes.(*Endpoints).resolvePodRef in discovery/kubernetes/endpoints.go 2.50TB 2.66TB (flat, cum) 22.28% of Total . . 447:func (e *Endpoints) resolvePodRef(ref *apiv1.ObjectReference) *apiv1.Pod { . . 448: if ref == nil || ref.Kind != "Pod" { . . 449: return nil . . 450: } 2.50TB 2.50TB 451: p := &apiv1.Pod{} . . 452: p.Namespace = ref.Namespace . . 453: p.Name = ref.Name . . 454: . 156.31GB 455: obj, exists, err := e.podStore.Get(p) . . 456: if err != nil { . . 457: level.Error(e.logger).Log("msg", "resolving pod ref failed", "err", err) . . 458: return nil . . 459: } . . 460: if !exists { ``` This is some low hanging fruit that we can easily optimize. The key of an object has format "namespace/name" so generate that inside of Prometheus itself and use pooling. ``` goos: linux goarch: amd64 pkg: github.com/prometheus/prometheus/discovery/kubernetes cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz │ olddisc │ newdisc │ │ sec/op │ sec/op vs base │ ResolvePodRef-16 516.3n ± 17% 289.5n ± 7% -43.92% (p=0.000 n=10) │ olddisc │ newdisc │ │ B/op │ B/op vs base │ ResolvePodRef-16 1168.00 ± 0% 24.00 ± 0% -97.95% (p=0.000 n=10) │ olddisc │ newdisc │ │ allocs/op │ allocs/op vs base │ ResolvePodRef-16 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹ ¹ all samples are equal ``` Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-10-28 12:12:40 +02:00
3Juhwan	685d6d169f	refactor: reorder fields in defaultSDConfig initialization for consistency ... Signed-off-by: 3Juhwan <13selfesteem91@naver.com>	2024-10-28 10:40:49 +01:00
George Krajcsovits	eb3b349024	fix(nhcb): created timestamp fails when keeping classic histograms (#15218) ... The wrong source was used to return the created timestamp, leading to index out of bound panic. One line fix. Refactor the requirement test to be generic and be able to test OpenMetrics and Prom parsers as well. There are some differencies in what the parsers support, the Prom parser doesn't have created timestamp. The protobuf parser uses different formatting to identify the metric for the scrape loop. Each parser represents the sample timestamp differently. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-28 08:31:43 +01:00
Pedro Tanaka	bab587b9dc	Agent: allow for ingestion of CT samples (#15124) ... * Remove unused option from HeadOptions Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Improve docs for appendable() method in head appender Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Ingest CT (float) samples in Agent DB Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * allow for ingestion of CT native histogram Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * adding some verification for ct ts Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Validating CT histogram before append and add newly created series to pending series Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * checking the wal for written samples Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Checking for samples in test Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * adding case for validations Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * fixing comparison when dedupelabels is enabled Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * unite tests, use table testing Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Implement CT related methods in timestampTracker for write storage Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * adding error case to test Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * removing unused fields Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Updating lastTs for series when adding CT to invalidate duplicates Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * making sure that updating the lastTS wont cause OOO later on in Commit(); Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> --------- Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>	2024-10-27 01:06:34 +01:00
Jan Fajerski	3acb3144fe	update CHANGELOG ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-10-26 09:03:10 +02:00
gopi	372b83d7b8	Documented that WAL can still be written after memory-snapshot-on-shutdown (#15179) ... Documented that WAL can still be written after memory-snapshot-on-shutdown - #10824 Co-authored-by: Björn Rabenstein <github@rabenste.in> Signed-off-by: gopi <gopi.singaravelan.k@gmail.com> --------- Signed-off-by: Gopi-eng2202 <gopi.singaravelan.k@gmail.com> Signed-off-by: gopi <gopi.singaravelan.k@gmail.com> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2024-10-25 21:40:15 +02:00
Ayoub Mrini	93db81dd3d	Merge pull request #14983 from machine424/dopp ... fix(storage/mergeQuerier): fix a data race	2024-10-25 18:34:51 +02:00
Ayoub Mrini	42a30a0c47	Merge pull request #15217 from CharlieTLe/support-int-exemplars ... fix(otlptranslator/prometheusremotewrite): support int exemplar value type	2024-10-25 18:29:35 +02:00
Charlie Le	d87f7440ca	support int exemplar value type ... When the exemplar type is an int, it incorrectly gets converted to a 0 when DoubleValue() is called on the exemplar. This adds a check to ensure that the value is converted properly based on the type. Signed-off-by: Charlie Le <charlie_le@apple.com>	2024-10-25 08:47:54 -07:00
Jan Fajerski	38fd48e6b5	v2.55.0 ... -----BEGIN SSH SIGNATURE----- U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgX42TrpDUXJbbi9yZ3hs6cDg+kz G6d3nAlAb2XQInrEgAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5 AAAAQGoSEKIFT/BfavtG2qW9n7NYonNQk/9r6gCLvxln9elt1hiY0ZGcwRhm1QNx6FotxJ Y3LB9dt4s5akB3fOPkYwc= -----END SSH SIGNATURE----- Merge tag 'v2.55.0' into release-3.0.0-rc.0 v2.55.0	2024-10-25 14:16:22 +02:00
Jan Fajerski	f131cdd4c5	3.0 migration guide (#15099) ... * docs: 2 to 3 migration guide Signed-off-by: Jan Fajerski <jfajersk@redhat.com> * docs/stability: add 3.0 section Signed-off-by: Jan Fajerski <jfajersk@redhat.com> * docs/migration: details on enabling legacy name validation Signed-off-by: Owen Williams <owen.williams@grafana.com>\ * migration: add log format and `le` normalization Signed-off-by: Jan Fajerski <jfajersk@redhat.com> * migration: add new enable_http2 default for remote write Signed-off-by: Jan Fajerski <jfajersk@redhat.com> --------- Signed-off-by: Jan Fajerski <jfajersk@redhat.com> Signed-off-by: Owen Williams <owen.williams@grafana.com> Co-authored-by: Owen Williams <owen.williams@grafana.com>	2024-10-25 12:30:13 +02:00
Jan Fajerski	24a10528ac	Merge pull request #15205 from tjhop/chore/slog-fixes ... slog: various fixes	2024-10-25 11:11:46 +02:00
Jan Fajerski	07d01a9e0c	Merge pull request #15219 from jan--f/rw-default-http2-off ... [CHANGE] Remote-write: default enable_http2 to false	2024-10-25 11:07:55 +02:00
TJ Hoplock	4f9e4dc016	ref: remove unused deduper log wrapper methods ... I used these wrapper methods during initial development of the custom handler that the deduper now implements. Since the deduper implements slog.Handler and can be used directly as a logger, these wrapper methods are no longer needed. Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2024-10-24 22:31:37 -04:00
TJ Hoplock	b602393473	fix: avoid data race in log deduper ... This change should have been included in the initial prometheus slog conversion, but I must've lost track of it in all the rebases involved in that PR. This changes the dedupe logger so that the only method that needs to use the lock is the `Handle()` method that actually interacts with the deduplication map. Ex: ``` ================== WARNING: DATA RACE Write at 0x00c000518bc0 by goroutine 29481: github.com/prometheus/prometheus/util/logging.(*Deduper).WithAttrs() /home/tjhop/go/src/github.com/prometheus/prometheus/util/logging/dedupe.go:89 +0xef log/slog.(*Logger).With() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/log/slog/logger.go:132 +0x106 github.com/prometheus/prometheus/storage/remote.NewQueueManager() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/queue_manager.go:483 +0x7a9 github.com/prometheus/prometheus/storage/remote.(*WriteStorage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/write.go:201 +0x102c github.com/prometheus/prometheus/storage/remote.(*Storage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:92 +0xfd github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.func1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:172 +0x3e4 github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.gowrap1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:174 +0x41 Previous read at 0x00c000518bc0 by goroutine 31261: github.com/prometheus/prometheus/util/logging.(*Deduper).Handle() /home/tjhop/go/src/github.com/prometheus/prometheus/util/logging/dedupe.go:82 +0x2b1 log/slog.(*Logger).log() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/log/slog/logger.go:257 +0x228 log/slog.(*Logger).Error() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/log/slog/logger.go:230 +0x3d4 github.com/prometheus/prometheus/tsdb/wlog.(*Watcher).loop() /home/tjhop/go/src/github.com/prometheus/prometheus/tsdb/wlog/watcher.go:254 +0x2db github.com/prometheus/prometheus/tsdb/wlog.(*Watcher).Start.gowrap1() /home/tjhop/go/src/github.com/prometheus/prometheus/tsdb/wlog/watcher.go:227 +0x33 Goroutine 29481 (running) created at: github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:164 +0xe4 testing.tRunner() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/testing/testing.go:1690 +0x226 testing.(*T).Run.gowrap1() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/testing/testing.go:1743 +0x44 Goroutine 31261 (running) created at: github.com/prometheus/prometheus/tsdb/wlog.(*Watcher).Start() /home/tjhop/go/src/github.com/prometheus/prometheus/tsdb/wlog/watcher.go:227 +0x177 github.com/prometheus/prometheus/storage/remote.(*QueueManager).Start() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/queue_manager.go:934 +0x304 github.com/prometheus/prometheus/storage/remote.(*WriteStorage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/write.go:232 +0x151b github.com/prometheus/prometheus/storage/remote.(*Storage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:92 +0xfd github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.func1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:172 +0x3e4 github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.gowrap1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:174 +0x41 ================== --- FAIL: TestWriteStorageApplyConfigsDuringCommit (2.26s) testing.go:1399: race detected during execution of test FAIL FAIL github.com/prometheus/prometheus/storage/remote 68.321s ``` Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2024-10-24 22:30:38 -04:00