Commit graph

1392 commits

Author SHA1 Message Date
Nicolás Pazos 55c99d4efd fields rewording in handler
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos ba3422df1f fix NewWriteClient and change new flags wording
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
alexgreenbank 7dc2002ca9 gofmt
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
alexgreenbank 3f33175e0b Remove config, update proto
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
alexgreenbank d1d80b8bed Add 1.1 version handling code
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos 98a0d00eb2 cleanup: remove hardcoded fake url for testing
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos 26673bcbde Use github.com/golang/snappy
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos c710e19f9c Improve sender benchmarks and some allocations
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos 9b4fbd9552 fix build
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos e70780ca86 refactor queue manager code to remove some duplication
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos c7a77af3e2 add sender-side tests and fix failing ones
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos e909eaea31 fix: queue manager to include float histograms in new requests
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos 2e57d7e645 refactor out common code between write methods
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos 407e596ce3 add basic reduce remote write handler benchmark
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos 0b421384f3 fix mocks and fixture
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos 12de4c4a62 remove some comented code
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos 5f5272e69d no-brainer copypaste but more performance write support
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:54 -03:00
Nicolás Pazos ab7c96a30e add new proto support on receiver end
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:09 -03:00
Nicolás Pazos 005ba7ac97 tests and new -> original proto mapping util
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:09 -03:00
Callum Styan 91bdd93956 Add new test client
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:09 -03:00
Callum Styan ce1e2ad0b9 Implement code paths for new proto format
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:09 -03:00
Callum Styan 0768e55597 add lookup table struct that is used to build the symbol table in new
write request format

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:09 -03:00
Callum Styan 81940007ff add decode function for new write request proto
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:09 -03:00
Callum Styan 4c4b9aa471 replace snappy encoding library
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
2023-12-19 14:27:09 -03:00
Björn Rabenstein 775de1a3bd
Merge pull request #13276 from fpetkovski/reuse-float-histograms
Reuse float histogram objects
2023-12-13 13:43:01 +01:00
Filip Petkovski ea356c472e
Add comment on SampleRingIterator methods
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-13 08:35:02 +01:00
Filip Petkovski bb8363dbb3
Add comment on SampleRingIterator
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-13 08:30:02 +01:00
Filip Petkovski 48df9fc020
Export SampleRingIterator
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-11 11:18:08 +01:00
Arthur Silva Sens 5082655392
Append Created Timestamps (#12733)
* Append created timestamps.

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Log when created timestamps are ignored

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Proposed changes to Append CT PR.

Changes:

* Changed textparse Parser interface for consistency and robustness.
* Changed CT interface to be more explicit and handle validation.
* Simplified test, change scrapeManager to allow testability.
* Added TODOs.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Updates.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Refactor head_appender test

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Fix linter issues

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Use model.Sample in head appender test

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

---------

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2023-12-11 08:43:42 +00:00
Filip Petkovski e2a9f8ac0f
Reuse float histogram objects
This commit reduces the memory needed to query native histogram objects
by reusing existing HPoint instances.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-11 08:24:58 +01:00
Filip Petkovski 10a82f87fd
Enable reusing memory when converting between histogram types
The 'ToFloat' method on integer histograms currently allocates new memory
each time it is called.

This commit adds an optional *FloatHistogram parameter that can be used
to reuse span and bucket slices. It is up to the caller to make sure the
input float histogram is not used anymore after the call.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-08 10:22:59 +01:00
Matthieu MOREL 9c4782f1cc
golangci-lint: enable testifylint linter (#13254)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-12-07 11:35:01 +00:00
Oleksandr Redko 2a75604f8e
Enable default revive rules (#13068)
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-11-29 17:23:34 +00:00
Fiona Liao 5bee0cfce2
Change ChunkReader.Chunk() to ChunkOrIterable()
The ChunkReader interface's Chunk() has been changed to ChunkOrIterable(). 

This is a precursor to OOO native histogram support - with OOO native histograms, the chunks.Meta passed to Chunk() can result in multiple chunks being returned rather than just a single chunk (e.g. if oooMergedChunk has a counter reset in the middle). 

To support this, ChunkOrIterable() requires either a single chunk or an iterable to be returned. If an iterable is returned, the caller has the responsibility of converting the samples from the iterable into possibly multiple chunks. The OOOHeadChunkReader now returns an iterable rather than a chunk to prepare for the native histograms case. Also as a beneficial side effect, oooMergedChunk and boundedChunk has been simplified as they only need to implement the Iterable interface now, not the full Chunk interface.

---------

Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2023-11-28 11:14:29 +01:00
Bryan Boreham a2d5c02298
Merge pull request #13084 from charleskorn/concatenatingchunkiterator
Fix issue where `concatenatingChunkIterator` can obscure errors
2023-11-22 08:16:39 +00:00
Goutham 3048a88ae7
Add suffixes
Older version already did that. This upgrade needed manual opt-in

Signed-off-by: Goutham <gouthamve@gmail.com>
2023-11-15 15:52:18 +01:00
Goutham a99f48cc9f
Bump OTel Collector dependency to v0.88.0
I initially didn't copy the otlptranslator/prometheus folder because I
assumed it wouldn't get changes. But it did. So this PR fixes that and
updates the Collector version.

Supersedes: https://github.com/prometheus/prometheus/pull/12809

Signed-off-by: Goutham <gouthamve@gmail.com>
2023-11-15 15:18:14 +01:00
machine424 413b713aa8
remote/storage.go: adjust Storage.Notify() to avoid a race condition with Storage.ApplyConfig()
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2023-11-14 10:07:45 +01:00
machine424 08c17df244
remote/storage.go: add a test to highlight a race condition
between Storage.Notify() and Storage.ApplyConfig()

see https://github.com/prometheus/prometheus/issues/12747

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2023-11-13 13:23:53 +01:00
machine424 0996b78326
remote_write: add a unit test to make sure the write client sends
the extra http headers as expected

This will help letting prometheus off the hook from situations like
https://github.com/prometheus/prometheus/issues/13030

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2023-11-09 15:56:48 +01:00
Julien Pivotto cf01ec2119
Merge pull request #13091 from mmorel-35/errorlint/util
util: use Go standard errors package
2023-11-07 21:07:25 -06:00
Linas Medziunas 1f8aea11d6 Move histogram validation code to model/histogram
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-03 16:17:24 +02:00
Linas Medziunas 1cd6c1cde5 ValidateHistogram: strict Count check in absence of NaNs
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-03 16:17:24 +02:00
Matthieu MOREL fe057fc60d use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-03 07:26:31 +00:00
Charles Korn 8274e248ad
Fix issue where concatenatingChunkIterator can obscure errors.
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-11-02 15:29:09 +11:00
Björn Rabenstein a43669e611
Merge pull request #12928 from alexandear/ci-enable-godot
ci(lint): enable godot; append dot at the end of comments
2023-11-01 17:15:41 +01:00
Charles Korn 5184368db6
Fix issue where chainSampleIterator can obscure errors (#13006)
* Fix issue where `chainSampleIterator` can obscure errors

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Address PR feedback.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-10-31 22:35:17 +00:00
Julien Pivotto f568221610
Merge pull request #13057 from prometheus/release-2.48
Merge release-2.48 back into main
2023-10-31 15:24:39 -04:00
Oleksandr Redko fa90ca46e5 ci(lint): enable godot; append dot at the end of comments
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-10-31 19:53:38 +02:00
beorn7 4696b46dd5 storage: Fix mixed samples handling in sampleRing
Two issues are fixed here, that lead to the same problem:

1. If `newSampleRing` is called with an unknown ValueType including
   ValueNone, we have initialized the interface buffer (`iBuf`).
   However, we would still use a specialized buffer for the first
   sample, opportunistically assuming that we might still not
   encounter mixed samples and we should go down the more efficient
   road.

2. If the `sampleRing` is `reset`, we leave all buffers alone,
   including `iBuf`, which is generally fine, but not for `iBuf`, see
   below.

In both cases, `iBuf` already contains values, but we will fill one of
the specialized buffers first. Once we then actually encounter mixed
samples, the content of the specialized buffer is copied into `iBuf`
using `append`. That's by itself the right idea because `iBuf` might
be `nil`, and even if not, it might or might not have the right
capacity. However, this approach assumes that `iBuf` is empty, or more
precisely has a length of zero.

This commit makes sure that `iBuf` does not get needlessly initialized
in `newSampleRing` and that it is emptied upon `reset`.

A test case is added to demonstrate both issues above.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-10-31 16:18:09 +01:00