1506 commits

Author	SHA1	Message	Date
🌲 Harry 🌊 John 🏔	f9bc50b247	storage: Implement limit in mergeGenericQuerier ... Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>	2024-11-07 09:08:23 -08:00
Matthieu MOREL	af1a19fc78	enable errorf rule from perfsprint linter ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2024-11-06 16:50:36 +01:00
Arve Knudsen	7c4f877881	otlptranslator: Harmonize non-UTF8 sanitization w/ naming rules. (#15314) ... * otlptranslator: Harmonize non-UTF8 sanitization w/ naming rules. Harmonize non-UTF8 sanitization w/ Prometheus naming rules. --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-11-03 12:56:54 +00:00
Arthur Silva Sens	1a22b1d846	bugfix: Fix otlp translator switching colons to undescores (#15251) ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-11-03 13:15:49 +01:00
Arthur Silva Sens	034d2b24bc	Fix typos in tests (#15312) ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-11-02 15:38:51 +01:00
Bartlomiej Plotka	dd4eb4590d	Merge pull request #15222 from jmichalek132/jm-add-tcp-events-to-tracing ... tracing: add tcp events to remote store span	2024-11-01 14:26:15 +01:00
Arthur Silva Sens	ea06f1a1d1	bugfix: Fix otlp translation of foreign characters ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-10-30 19:29:31 -03:00
Arthur Silva Sens	8588289c24	otlp translator: Add test showing bugs ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-10-30 19:28:53 -03:00
Juraj Michalek	76ff12b32a	chore: only create span events ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
Juraj Michalek	7ecdb55b02	chore: use otelhttptrace instead ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
Juraj Michalek	3c1ffbb2fd	chore: added idleTimeMs and fixed linting issues ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
Juraj Michalek	2e7c739d44	chore: add tcp events to remote store span ... Signed-off-by: Juraj Michalek <juraj.michalek132@gmail.com>	2024-10-30 09:41:38 +01:00
shenpengfeng	a44db5f784	chore: fix function name in comment ... Signed-off-by: shenpengfeng <xinhangzhou@icloud.com>	2024-10-29 17:58:44 +08:00
Pedro Tanaka	bab587b9dc	Agent: allow for ingestion of CT samples (#15124) ... * Remove unused option from HeadOptions Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Improve docs for appendable() method in head appender Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Ingest CT (float) samples in Agent DB Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * allow for ingestion of CT native histogram Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * adding some verification for ct ts Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Validating CT histogram before append and add newly created series to pending series Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * checking the wal for written samples Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Checking for samples in test Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * adding case for validations Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * fixing comparison when dedupelabels is enabled Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * unite tests, use table testing Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Implement CT related methods in timestampTracker for write storage Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * adding error case to test Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * removing unused fields Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * Updating lastTs for series when adding CT to invalidate duplicates Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> * making sure that updating the lastTS wont cause OOO later on in Commit(); Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com> --------- Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>	2024-10-27 01:06:34 +01:00
Ayoub Mrini	93db81dd3d	Merge pull request #14983 from machine424/dopp ... fix(storage/mergeQuerier): fix a data race	2024-10-25 18:34:51 +02:00
Charlie Le	d87f7440ca	support int exemplar value type ... When the exemplar type is an int, it incorrectly gets converted to a 0 when DoubleValue() is called on the exemplar. This adds a check to ensure that the value is converted properly based on the type. Signed-off-by: Charlie Le <charlie_le@apple.com>	2024-10-25 08:47:54 -07:00
Vanshika	cccbe72514	TSDB: Fix some edge cases when OOO is enabled (#14710) ... Fix some edge cases when OOO is enabled Signed-off-by: Vanshikav123 <vanshikav928@gmail.com> Signed-off-by: Vanshika <102902652+Vanshikav123@users.noreply.github.com> Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com> Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>	2024-10-23 17:34:28 +02:00
machine424	cebcdce78a	fix(storage/mergeQuerier): copy the matcjers slice before passing it to queriers as ... some of them may alter it. Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-10-22 14:08:47 +02:00
machine424	eb523a6b29	fix(storage/mergeQuerier): add a reproducer for data race that occurs when one of the queriers alters the passed matchers and propose a fix ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-10-22 14:08:46 +02:00
Björn Rabenstein	5e987e7c7d	Merge pull request #15092 from jan--f/clarify-storage-contract ... storage: require selectors to always return matching results	2024-10-09 13:07:59 +02:00
TJ Hoplock	6ebfbd2d54	chore!: adopt log/slog, remove go-kit/log ... For: #14355 This commit updates Prometheus to adopt stdlib's log/slog package in favor of go-kit/log. As part of converting to use slog, several other related changes are required to get prometheus working, including: - removed unused logging util func `RateLimit()` - forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger - move some of the json file logging functionality to use prom/common package functionality - refactored some of the new json file logging for scraping - changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers - updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition - added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2024-10-07 15:58:50 -04:00
Matthieu MOREL	ab64966e9d	fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094) ... * fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" --------- Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-10-06 16:35:29 +00:00
Jan Fajerski	563bfdd384	storage: require selectors to always return matching results ... Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-10-04 12:10:16 +02:00
György Krajcsovits	44ebbb8458	Fix missing histogram copy in sampleRing ... The specialized version of sample add to the ring: func addH(s hSample, buf []hSample, r *sampleRing) []hSample func addFH(s fhSample, buf []fhSample, r *sampleRing) []fhSample already correctly copy histogram samples from the reused hReader, fhReader buffers, but the generic version does not. This means that the data is overwritten on the next read if the sample ring has seen histogram and float samples at the same time and switched to generic mode. The `genericAdd` function (which was commented anyway) is by now quite different from the specialized functions so that this commit deletes it. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-02 13:57:28 +02:00
György Krajcsovits	b5479831b8	Unit test for regression in rate vs float and histogram mixup ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-10-02 13:56:33 +02:00
Ayoub Mrini	105ab2e95a	fix(test): adjust defer invocations (#14996) ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-09-27 17:13:51 +01:00
Arthur Silva Sens	6bd9b1a7cc	Histogram CT Zero ingestion ... Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>	2024-09-26 11:29:22 -03:00
Callum Styan	005bd33fe2	support v2 proto for BenchmarkSampleSend (#14935) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	2024-09-19 15:38:26 -07:00
Callum Styan	c328d5fc88	fix rwv2 build write request benchmark, also change how the memory usage (#14925) ... is reported for these benchmarks to more accurately reflect what's actually allocated Signed-off-by: Callum Styan <callumstyan@gmail.com>	2024-09-18 07:04:10 +01:00
Antoine Pultier	d90d0976b5	fix(bstream/writeByte): ensure it appends only one byte (#14854) ... fix(bstream/writeByte): ensure it appends only one byte Signed-off-by: Antoine Pultier <antoine.pultier@sintef.no>	2024-09-17 16:28:33 +02:00
Carrie Edwards	14e3c05ce8	tsdb: Add support for ingestion of out-of-order native histogram samples (#14546) ... Add support for ingesting OOO native histograms * Add flag for enabling and disabling OOO native histogram ingestion * Update OOO querying tests to include native histogram samples * Add OOO head tests * Add test for OOO native histogram counter reset headers Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored by: Carrie Edwards <edwrdscarrie@gmail.com> Co-authored by: Jeanette Tan <jeanette.tan@grafana.com> Co-authored by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored by: Fiona Liao <fiona.liao@grafana.com>	2024-09-17 11:19:06 +02:00
Harry John	919dc0cbc6	storage: Update LabelQuerier interface to return sorted label values (#14849) ... * Change LabelQuerier.LabelValues() to return sorted values --------- Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>	2024-09-17 08:55:02 +02:00
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868) ... * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	2024-09-10 22:32:03 +02:00
Arve Knudsen	4fc562f9e7	OTLP: Support context cancellation/timeout during translation (#14612) ... * OTLP: Support context cancellation/timeout during translation --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-09-08 17:13:40 +02:00
György Krajcsovits	a693dd19f2	Fix: chunkenc.MockSeriesIterator ... Starts its index from 0 , but users call Next() before first sample so it needs to start from -1 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-30 16:44:36 +02:00
Arve Knudsen	bc6c2c5d35	OTLP Receiver: Add tests (#14764) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-30 11:30:57 +02:00
Bryan Boreham	f90c7a11d1	[REFACTOR] OTLP translator: simplify time conversion ... We don't need multiple levels of abstraction to convert nanoseconds to milliseconds. We do benefit from tests, however. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-08-29 12:00:01 +01:00
Owen Williams	9da75328ea	fix(utf8): ensure correct validation when legacy mode turned on (#14736) ... fix(utf8): ensure correct validation when legacy mode turned on This depends on the included update of the prometheus/common dependency. --------- Signed-off-by: Owen Williams <owen.williams@grafana.com>	2024-08-28 17:15:42 +02:00
Justin Lei	3a82cd5a7e	Add streaming remote read to ReadClient (#11379) ... * Add streaming remote read to ReadClient Signed-off-by: Justin Lei <justin.lei@grafana.com> * Apply suggestions from code review Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Justin Lei <justin.lei@grafana.com> * Remote read instrumentation tweaks Signed-off-by: Justin Lei <lei.justin@gmail.com> * Minor cleanups Signed-off-by: Justin Lei <lei.justin@gmail.com> * In-line handleChunkedResponse Signed-off-by: Justin Lei <lei.justin@gmail.com> * Fix lints Signed-off-by: Justin Lei <lei.justin@gmail.com> * Explicitly call cancel() when needed Signed-off-by: Justin Lei <lei.justin@gmail.com> * Update chunkedSeries, chunkedSeriesIterator for new interfaces Signed-off-by: Justin Lei <lei.justin@gmail.com> * Adapt remote.chunkedSeries to use prompb.ChunkedSeries Signed-off-by: Justin Lei <lei.justin@gmail.com> * Fix lint Signed-off-by: Justin Lei <lei.justin@gmail.com> --------- Signed-off-by: Justin Lei <justin.lei@grafana.com> Signed-off-by: Justin Lei <lei.justin@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	2024-08-28 07:23:54 +01:00
Devin Trejo	d4994e5bc4	fix: Remote-write-reciever returns 4xx when request contains a time series with duplicate labels. (#14716) ... Signed-off-by: Devin Trejo <dtrejo@palantir.com>	2024-08-23 15:15:27 -07:00
beorn7	0f760f63dd	lint: Revamp our linting rules, mostly around doc comments ... Several things done here: - Set `max-issues-per-linter` to 0 so that we actually see all linter warnings and not just 50 per linter. (As we also set `max-same-issues` to 0, I assume this was the intention from the beginning.) - Stop using the golangci-lint default excludes (by setting `exclude-use-default: false`. Those are too generous and don't match our style conventions. (I have re-added some of the excludes explicitly in this commit. See below.) - Re-add the `errcheck` exclusion we have used so far via the defaults. - Exclude the signature requirement `govet` has for `Seek` methods because we use non-standard `Seek` methods a lot. (But we keep other requirements, while the default excludes completely disabled the check for common method segnatures.) - Exclude warnings about missing doc comments on exported symbols. (We used to be pretty adamant about doc comments, but stopped that at some point in the past. By now, we have about 500 missing doc comments. We may consider reintroducing this check, but that's outside of the scope of this commit. The default excludes of golangci-lint essentially ignore doc comments completely.) - By stop using the default excludes, we now get warnings back on malformed doc comments. That's the most impactful change in this commit. It does not enforce doc comments (again), but _if_ there is a doc comment, it has to have the recommended form. (Most of the changes in this commit are fixing this form.) - Improve wording/spelling of some comments in .golangci.yml, and remove an outdated comment. - Leave `package-comments` inactive, but add a TODO asking if we should change that. - Add a new sub-linter `comment-spacings` (and fix corresponding comments), which avoids missing spaces after the leading `//`. Signed-off-by: beorn7 <beorn@grafana.com>	2024-08-22 17:36:11 +02:00
Bryan Boreham	95b3c49836	[TESTS] remote-write: fix slice init length (#14702) ... fix: fix slice init length	2024-08-22 12:53:45 +01:00
Arve Knudsen	b50c5d42fe	OTLP receiver: Warn when encountering invalid exponential histograms ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-21 16:43:16 +02:00
cuishuang	2110661121	fix: fix slice init length ... Signed-off-by: cuishuang <imcusg@gmail.com>	2024-08-21 18:15:25 +08:00
Arve Knudsen	3a78e76282	Upgrade golangci-lint to v1.60.1 ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-18 12:13:25 +02:00
Björn Rabenstein	1daf7cdd62	Merge pull request #14626 from cuiweiyuan/main ... chore: fix some function names	2024-08-15 11:46:21 +02:00
cuiweiyuan	1800af54f0	chore: fix some function names ... Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>	2024-08-15 13:57:21 +08:00
Arve Knudsen	5fd66ba855	Merge pull request #14510 from aknuds1/arve/add-normalize-name-tests ... otlptranslator: Add tests for BuildCompliantName	2024-08-09 18:21:16 +02:00
Oleg Zaytsev	0833d2a230	Fix appendable: check whether last val was a histogram (#14613) ... * Fix appendable: check whether last val was a histogram When appending a float, we were checking whether lastValue was equal to current value, but we didn't check whether last value was a float value. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	2024-08-07 15:02:59 +02:00
George Krajcsovits	00ab05c3b9	Native histograms: fix spurios counter reset when merging recoded chunk to normal chunk (#14513) ... * chunkenc: allow missing empty buckets on histogram append Allow appending to chunks when the histogram to be added is missing some buckets, but the missing buckets are empty in the chunk. For example bucket at index 5 is present in the chunk, but its value is 0 and the new histogram doesn't have a bucket at index 5. This fixes an issue of merging chunks where one chunk was recoded to retroactively have some empty buckets in all the histograms and we are merging in a histogram that doesn't have the empty bucket (because it was not recoded yet). The operation alters the histogram that is being added, however this has already been the case when appending gauge histograms. Thus the test TestHistogramSeriesToChunks in storage package is changed to explicitly test what happened to the appended histogram - Compact(0) call is removed. The new expandIntSpansAndBuckets and expandFloatSpansAndBuckets functions are a merge of expandSpansForward and counterResetInAnyBucket and counterResetInAnyFloatBucket. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-08-01 09:22:32 +02:00