14474 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Juraj Michalek	3c1ffbb2fd	chore: added idleTimeMs and fixed linting issues ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
Juraj Michalek	2e7c739d44	chore: add tcp events to remote store span ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
Julius Volz	c861b31b72	Support UTF-8 metric names and labels in web UI ... Fixes most of https://github.com/prometheus/prometheus/issues/15202 This should address all areas of the UI except for the autocompletion in the codemirror-promql text editor. The strategy here is that any time we print or internally serialize (like for the PromLens tree view) either a metric name or a label name as part of a selector or in other relevant parts of PromQL, we check whether it contains characters beyond what was previously supported, and if so, quote and escape it. In the case of metric names, we also have to move them from the beginning of the selector into the curly braces. Signed-off-by: Julius Volz <julius.volz@gmail.com>	2024-10-29 20:22:52 +01:00
Ganesh Vernekar	d0eecb1223	Merge pull request #15239 from colega/revert-14975-process-mempostings-delete-with-gomaxprocs-workers ... Revert "Process `MemPostings.Delete()` with `GOMAXPROCS` workers"	2024-10-29 12:27:43 -04:00
Oleg Zaytsev	ba11a55df4	Revert "Process MemPostings.Delete() with GOMAXPROCS workers" ... Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	2024-10-29 17:13:40 +01:00
Jan Fajerski	e2f55c34c9	fix CHANGELOG formatting and add entry for #14694 ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-10-29 14:41:50 +01:00
Nicolas Takashi	b6c538972c	[REFACTORY] simplify appender commit (#15112) ... * [REFACTOR] simplify appender commit Signed-off-by: Nicolas Takashi <nicolas.tcs@hotmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-10-29 12:34:02 +00:00
Ayoub Mrini	350e0d5bc1	Merge pull request #15228 from shenpengfeng/main ... chore: fix function name in comment	2024-10-29 12:12:59 +01:00
shenpengfeng	a44db5f784	chore: fix function name in comment ... Signed-off-by: shenpengfeng <xinhangzhou@icloud.com>	2024-10-29 17:58:44 +08:00
Arve Knudsen	706dcfeecf	tsdb.CircularExemplarStorage: Avoid racing (#15231) ... * tsdb.CircularExemplarStorage: Avoid racing --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-10-29 10:40:46 +01:00
Ayoub Mrini	559722dc68	Merge pull request #15206 from multani/fix-doc-formatting ... doc: fix formatting	2024-10-28 22:53:55 +01:00
Ayoub Mrini	99c0e68013	Merge pull request #15230 from GiedriusS/optimize_resolvepodref ... discovery/kubernetes: optimize resolvePodRef	2024-10-28 22:53:13 +01:00
Jan Fajerski	5408184452	react-app: bump version in lock file ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-10-28 17:10:34 +01:00
Jack Westbrook	7cda23ba32	fix(lezer-promql): fix missing types export in package.json (#15161) ... Signed-off-by: Jack Westbrook <jack.westbrook@gmail.com>	2024-10-28 17:05:10 +01:00
Giedrius Statkevičius	58fedb6b61	discovery/kubernetes: optimize more gets ... Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-10-28 17:17:37 +02:00
Giedrius Statkevičius	716fd5b11f	discovery/kubernetes: use namespacedName ... Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-10-28 16:19:56 +02:00
Giedrius Statkevičius	e452308e37	discovery/kubernetes: optimize resolvePodRef ... resolvePodRef is in a hot path: ``` ROUTINE ======================== github.com/prometheus/prometheus/discovery/kubernetes.(*Endpoints).resolvePodRef in discovery/kubernetes/endpoints.go 2.50TB 2.66TB (flat, cum) 22.28% of Total . . 447:func (e *Endpoints) resolvePodRef(ref *apiv1.ObjectReference) *apiv1.Pod { . . 448: if ref == nil || ref.Kind != "Pod" { . . 449: return nil . . 450: } 2.50TB 2.50TB 451: p := &apiv1.Pod{} . . 452: p.Namespace = ref.Namespace . . 453: p.Name = ref.Name . . 454: . 156.31GB 455: obj, exists, err := e.podStore.Get(p) . . 456: if err != nil { . . 457: level.Error(e.logger).Log("msg", "resolving pod ref failed", "err", err) . . 458: return nil . . 459: } . . 460: if !exists { ``` This is some low hanging fruit that we can easily optimize. The key of an object has format "namespace/name" so generate that inside of Prometheus itself and use pooling. ``` goos: linux goarch: amd64 pkg: github.com/prometheus/prometheus/discovery/kubernetes cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz │ olddisc │ newdisc │ │ sec/op │ sec/op vs base │ ResolvePodRef-16 516.3n ± 17% 289.5n ± 7% -43.92% (p=0.000 n=10) │ olddisc │ newdisc │ │ B/op │ B/op vs base │ ResolvePodRef-16 1168.00 ± 0% 24.00 ± 0% -97.95% (p=0.000 n=10) │ olddisc │ newdisc │ │ allocs/op │ allocs/op vs base │ ResolvePodRef-16 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹ ¹ all samples are equal ``` Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-10-28 12:12:40 +02:00
3Juhwan	685d6d169f	refactor: reorder fields in defaultSDConfig initialization for consistency ... Signed-off-by: 3Juhwan <13selfesteem91@naver.com>	2024-10-28 10:40:49 +01:00
György Krajcsovits	eafe72a0d0	perf(nhcb): optimize away most allocations in convertnhcb ... In general aim for the happy case when the exposer lists the buckets in ascending order. Use Compact(2) to compact the result of nhcb convert. This is more in line with how client_golang optimizes spans vs buckets. aef8aedb4b/prometheus/histogram.go (L1485) Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-28 08:34:54 +01:00
George Krajcsovits	eb3b349024	fix(nhcb): created timestamp fails when keeping classic histograms (#15218) ... The wrong source was used to return the created timestamp, leading to index out of bound panic. One line fix. Refactor the requirement test to be generic and be able to test OpenMetrics and Prom parsers as well. There are some differencies in what the parsers support, the Prom parser doesn't have created timestamp. The protobuf parser uses different formatting to identify the metric for the scrape loop. Each parser represents the sample timestamp differently. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-28 08:31:43 +01:00
Pedro Tanaka	bab587b9dc	Agent: allow for ingestion of CT samples (#15124) ... * Remove unused option from HeadOptions Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Improve docs for appendable() method in head appender Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Ingest CT (float) samples in Agent DB Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * allow for ingestion of CT native histogram Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * adding some verification for ct ts Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Validating CT histogram before append and add newly created series to pending series Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * checking the wal for written samples Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Checking for samples in test Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * adding case for validations Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * fixing comparison when dedupelabels is enabled Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * unite tests, use table testing Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Implement CT related methods in timestampTracker for write storage Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * adding error case to test Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * removing unused fields Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Updating lastTs for series when adding CT to invalidate duplicates Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * making sure that updating the lastTS wont cause OOO later on in Commit(); Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> --------- Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>	2024-10-27 01:06:34 +01:00
Jan Fajerski	3acb3144fe	update CHANGELOG ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-10-26 09:03:10 +02:00
gopi	372b83d7b8	Documented that WAL can still be written after memory-snapshot-on-shutdown (#15179) ... Documented that WAL can still be written after memory-snapshot-on-shutdown - #10824 Co-authored-by: Björn Rabenstein <github@rabenste.in> Signed-off-by: gopi <gopi.singaravelan.k@gmail.com> --------- Signed-off-by: Gopi-eng2202 <gopi.singaravelan.k@gmail.com> Signed-off-by: gopi <gopi.singaravelan.k@gmail.com> Co-authored-by: Björn Rabenstein <github@rabenste.in>	2024-10-25 21:40:15 +02:00
Ayoub Mrini	93db81dd3d	Merge pull request #14983 from machine424/dopp ... fix(storage/mergeQuerier): fix a data race	2024-10-25 18:34:51 +02:00
Ayoub Mrini	42a30a0c47	Merge pull request #15217 from CharlieTLe/support-int-exemplars ... fix(otlptranslator/prometheusremotewrite): support int exemplar value type	2024-10-25 18:29:35 +02:00
Charlie Le	d87f7440ca	support int exemplar value type ... When the exemplar type is an int, it incorrectly gets converted to a 0 when DoubleValue() is called on the exemplar. This adds a check to ensure that the value is converted properly based on the type. Signed-off-by: Charlie Le <charlie_le@apple.com>	2024-10-25 08:47:54 -07:00
Jan Fajerski	38fd48e6b5	v2.55.0 ... -----BEGIN SSH SIGNATURE----- U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgX42TrpDUXJbbi9yZ3hs6cDg+kz G6d3nAlAb2XQInrEgAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5 AAAAQGoSEKIFT/BfavtG2qW9n7NYonNQk/9r6gCLvxln9elt1hiY0ZGcwRhm1QNx6FotxJ Y3LB9dt4s5akB3fOPkYwc= -----END SSH SIGNATURE----- Merge tag 'v2.55.0' into release-3.0.0-rc.0 v2.55.0	2024-10-25 14:16:22 +02:00
Jan Fajerski	f131cdd4c5	3.0 migration guide (#15099) ... * docs: 2 to 3 migration guide Signed-off-by: Jan Fajerski <jfajersk@redhat.com> * docs/stability: add 3.0 section Signed-off-by: Jan Fajerski <jfajersk@redhat.com> * docs/migration: details on enabling legacy name validation Signed-off-by: Owen Williams <owen.williams@grafana.com>\ * migration: add log format and `le` normalization Signed-off-by: Jan Fajerski <jfajersk@redhat.com> * migration: add new enable_http2 default for remote write Signed-off-by: Jan Fajerski <jfajersk@redhat.com> --------- Signed-off-by: Jan Fajerski <jfajersk@redhat.com> Signed-off-by: Owen Williams <owen.williams@grafana.com> Co-authored-by: Owen Williams <owen.williams@grafana.com>	2024-10-25 12:30:13 +02:00
Jan Fajerski	24a10528ac	Merge pull request #15205 from tjhop/chore/slog-fixes ... slog: various fixes	2024-10-25 11:11:46 +02:00
Jan Fajerski	07d01a9e0c	Merge pull request #15219 from jan--f/rw-default-http2-off ... [CHANGE] Remote-write: default enable_http2 to false	2024-10-25 11:07:55 +02:00
TJ Hoplock	4f9e4dc016	ref: remove unused deduper log wrapper methods ... I used these wrapper methods during initial development of the custom handler that the deduper now implements. Since the deduper implements slog.Handler and can be used directly as a logger, these wrapper methods are no longer needed. Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2024-10-24 22:31:37 -04:00
TJ Hoplock	b602393473	fix: avoid data race in log deduper ... This change should have been included in the initial prometheus slog conversion, but I must've lost track of it in all the rebases involved in that PR. This changes the dedupe logger so that the only method that needs to use the lock is the `Handle()` method that actually interacts with the deduplication map. Ex: ``` ================== WARNING: DATA RACE Write at 0x00c000518bc0 by goroutine 29481: github.com/prometheus/prometheus/util/logging.(*Deduper).WithAttrs() /home/tjhop/go/src/github.com/prometheus/prometheus/util/logging/dedupe.go:89 +0xef log/slog.(*Logger).With() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/log/slog/logger.go:132 +0x106 github.com/prometheus/prometheus/storage/remote.NewQueueManager() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/queue_manager.go:483 +0x7a9 github.com/prometheus/prometheus/storage/remote.(*WriteStorage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/write.go:201 +0x102c github.com/prometheus/prometheus/storage/remote.(*Storage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:92 +0xfd github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.func1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:172 +0x3e4 github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.gowrap1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:174 +0x41 Previous read at 0x00c000518bc0 by goroutine 31261: github.com/prometheus/prometheus/util/logging.(*Deduper).Handle() /home/tjhop/go/src/github.com/prometheus/prometheus/util/logging/dedupe.go:82 +0x2b1 log/slog.(*Logger).log() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/log/slog/logger.go:257 +0x228 log/slog.(*Logger).Error() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/log/slog/logger.go:230 +0x3d4 github.com/prometheus/prometheus/tsdb/wlog.(*Watcher).loop() /home/tjhop/go/src/github.com/prometheus/prometheus/tsdb/wlog/watcher.go:254 +0x2db github.com/prometheus/prometheus/tsdb/wlog.(*Watcher).Start.gowrap1() /home/tjhop/go/src/github.com/prometheus/prometheus/tsdb/wlog/watcher.go:227 +0x33 Goroutine 29481 (running) created at: github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:164 +0xe4 testing.tRunner() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/testing/testing.go:1690 +0x226 testing.(*T).Run.gowrap1() /home/tjhop/.asdf/installs/golang/1.23.1/go/src/testing/testing.go:1743 +0x44 Goroutine 31261 (running) created at: github.com/prometheus/prometheus/tsdb/wlog.(*Watcher).Start() /home/tjhop/go/src/github.com/prometheus/prometheus/tsdb/wlog/watcher.go:227 +0x177 github.com/prometheus/prometheus/storage/remote.(*QueueManager).Start() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/queue_manager.go:934 +0x304 github.com/prometheus/prometheus/storage/remote.(*WriteStorage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/write.go:232 +0x151b github.com/prometheus/prometheus/storage/remote.(*Storage).ApplyConfig() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:92 +0xfd github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.func1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:172 +0x3e4 github.com/prometheus/prometheus/storage/remote.TestWriteStorageApplyConfigsDuringCommit.gowrap1() /home/tjhop/go/src/github.com/prometheus/prometheus/storage/remote/storage_test.go:174 +0x41 ================== --- FAIL: TestWriteStorageApplyConfigsDuringCommit (2.26s) testing.go:1399: race detected during execution of test FAIL FAIL github.com/prometheus/prometheus/storage/remote 68.321s ``` Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2024-10-24 22:30:38 -04:00
Jan Fajerski	7939eab77a	remote-write: change test default expected to http2 disabled ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-10-24 22:32:08 +02:00
Bryan Boreham	20fdc8f541	[CHANGE] Remote-write: default enable_http2 to false ... Remote-write creates several shards to parallelise sending, each with its own http connection. We do not want them all combined onto one socket by http2. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-10-24 22:27:06 +02:00
Ben Ye	99882eec3b	log last series labelset when hitting OOO series labels during compaction ... Signed-off-by: Ben Ye <benye@amazon.com>	2024-10-24 09:27:15 -07:00
Jesus Vazquez	3cb09acb21	Docs: Remove experimental note on out of order feature (#15215) ... Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>	2024-10-24 18:18:21 +02:00
George Krajcsovits	469573b13b	fix(nhcb): do not return nhcb from parse if exponential is present (#15209) ... From: https://github.com/prometheus/prometheus/pull/14978#discussion_r1800755481 Also encode the requirement table set in #13532 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-24 18:14:05 +02:00
Jonathan Ballet	7ca90e5729	doc: fix formatting ... Signed-off-by: Jonathan Ballet <jon@multani.info>	2024-10-24 08:55:23 +02:00
George Krajcsovits	2182b83271	feat(nhcb): implement created timestamp handling (#15198) ... Call through to the underlaying parser if we are not in a histogram and the entry is a series or exponential native histogram. Otherwise store and retrieve CT for NHCB. * fix(omparser): losing exemplars when CT is parsed Fixes: #15137 Ignore exemplars while peeking ahead during CT parsing. Simplify state reset with defer(). Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-24 07:38:58 +02:00
Vanshika	cccbe72514	TSDB: Fix some edge cases when OOO is enabled (#14710) ... Fix some edge cases when OOO is enabled Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> Signed-off-by: Vanshika <102902652+Vanshikav123@users.noreply.github.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>	2024-10-23 17:34:28 +02:00
Björn Rabenstein	7c7116fea8	Merge pull request #15176 from jhesketh/jhesketh/round ... Round function should ignore native histograms	2024-10-22 19:14:16 +02:00
George Krajcsovits	aa81210c8b	NHCB scrape: refactor state handling and speed up scrape test (#15193) ... * NHCB: scrape use state field and not booleans From comment https://github.com/prometheus/prometheus/pull/14978#discussion_r1800898724 Also make compareLabels read only and move storeLabels to the first processed classic histogram series. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Speed up TestConvertClassicHistogramsToNHCB 3x Reduce the startup time and timeouts Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * lint fix Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-22 17:49:25 +01:00
Björn Rabenstein	3bb5e28c6b	Merge pull request #15197 from prometheus/alexg/docs-issue-11570 ... docs: add keep_firing_for in alerting rules	2024-10-22 15:35:36 +02:00
George Krajcsovits	1b4e7f74e6	feat(tools): add debug printouts to rules unit testing (#15196) ... * promtool: Add debug flag for rule tests This makes it print out the tsdb state (both input_series and rules that are run) at the end of a test, making reasoning about tests much easier. Signed-off-by: David Leadbeater <dgl@dgl.cx> * Reuse generated test name from junit testing Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: David Leadbeater <dgl@dgl.cx> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: David Leadbeater <dgl@dgl.cx>	2024-10-22 15:24:36 +02:00
alexgreenbank	3afcda82be	docs: add keep_firing_for in alerting rules ... Signed-off-by: alexgreenbank <alex.greenbank@grafana.com>	2024-10-22 14:19:01 +01:00
machine424	cebcdce78a	fix(storage/mergeQuerier): copy the matcjers slice before passing it to queriers as ... some of them may alter it. Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-10-22 14:08:47 +02:00
machine424	eb523a6b29	fix(storage/mergeQuerier): add a reproducer for data race that occurs when one of the queriers alters the passed matchers and propose a fix ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-10-22 14:08:46 +02:00
Bryan Boreham	91d80252c3	Merge pull request #15194 from prometheus/make-release-2.55 ... Create release 2.55.0	2024-10-22 11:53:52 +01:00
Bryan Boreham	bb27c6b896	Create release 2.55.0 ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-10-22 09:31:02 +01:00
George Krajcsovits	ad4857de52	Merge pull request #14978 from prometheus/nhcb-scrape-impl ... feat: NHCB: convert classic histograms to nhcb in scrape MVP	2024-10-22 07:55:58 +02:00