prometheus/storage
sh0rez 5303e515af
Some checks are pending
buf.build / lint and publish (push) Waiting to run
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
remote/otlp: convert delta to cumulative (#15165)
What

Adds support for OTLP delta temporality to the OTLP endpoint.
This is done by calling the deltatocumulative processor from the OpenTelemetry collector during OTLP conversion.

Why

Delta conversion is a naturally stateful process, which requires careful request routing when operated inside a collector.
Prometheus is already stateful and doing the conversion in-server reduces the operational burden on the ingest architecture by only having one stateful component.

How

deltatocumulative is a OTel collector component that works as follows:

* pmetric.Metrics come from a receiver or in this case from the HTTP client
* It operates as an in-place update loop:
    * for each sample, if not delta, leave unmodified
    * if delta, do:
      * state += sample, where state is the in-memory sum of all previous samples
      * sample = state, sample value is now cumulative
    * this is supported for sums (counters), gauges, histograms (old histograms) and exponential histograms (native histograms)
If a series receives no new samples for 5m, its state is removed from memory


Performance

Delta performance is a stateful operation and the OTel code is not highly optimized yet, e.g. it locks the entire processor for each request. Nonetheless, care has been taken to mitigate those effects:

delta conversion is behind a feature flag. If disabled, no conversion code is ever invoked
if enabled, conversion is not invoked if request not actually contains delta samples. This leads to no measureable performance difference between default-cumulative to convert-cumulative (only cumulative, feature on/off)

Signed-off-by: sh0rez <me@shorez.de>
2025-01-14 11:33:31 -03:00
..
remote remote/otlp: convert delta to cumulative (#15165) 2025-01-14 11:33:31 -03:00
buffer.go chore: fix function name in comment 2024-10-29 17:58:44 +08:00
buffer_test.go Unit test for regression in rate vs float and histogram mixup 2024-10-02 13:56:33 +02:00
errors.go Fix appendable: check whether last val was a histogram (#14613) 2024-08-07 15:02:59 +02:00
errors_test.go Fix appendable: check whether last val was a histogram (#14613) 2024-08-07 15:02:59 +02:00
fanout.go TSDB: Fix some edge cases when OOO is enabled (#14710) 2024-10-23 17:34:28 +02:00
fanout_test.go fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094) 2024-10-06 16:35:29 +00:00
generic.go
interface.go enable errorf rule from perfsprint linter 2024-11-06 16:50:36 +01:00
interface_test.go Fix: chunkenc.MockSeriesIterator 2024-08-30 16:44:36 +02:00
lazy.go
memoized_iterator.go storage: add AtT method to MemoizedSeriesIterator 2024-07-15 14:09:02 +10:00
memoized_iterator_test.go storage: add AtT method to MemoizedSeriesIterator 2024-07-15 14:09:02 +10:00
merge.go storage: Implement limit in mergeGenericQuerier 2024-11-07 09:08:23 -08:00
merge_test.go fix TestOOOHeadChunkReader_Chunk on 32-bit 2024-12-16 10:45:07 -05:00
noop.go
secondary.go
series.go Unit test for regression in rate vs float and histogram mixup 2024-10-02 13:56:33 +02:00
series_test.go Upgrade to golangci-lint v1.62.0 (#15424) 2024-11-20 17:22:20 +01:00