Commit graph

286 commits

Author SHA1 Message Date
Björn Rabenstein 928d07e3bd
Merge branch 'main' into arve/typos
Signed-off-by: Björn Rabenstein <beorn@grafana.com>
2023-12-12 12:02:03 +01:00
Arthur Silva Sens 5082655392
Append Created Timestamps (#12733)
* Append created timestamps.

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Log when created timestamps are ignored

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Proposed changes to Append CT PR.

Changes:

* Changed textparse Parser interface for consistency and robustness.
* Changed CT interface to be more explicit and handle validation.
* Simplified test, change scrapeManager to allow testability.
* Added TODOs.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Updates.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Refactor head_appender test

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Fix linter issues

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

* Use model.Sample in head appender test

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>

---------

Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2023-12-11 08:43:42 +00:00
Filip Petkovski 9008271df5
Use cap to determine slice capacity
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-08 10:23:01 +01:00
Filip Petkovski 10a82f87fd
Enable reusing memory when converting between histogram types
The 'ToFloat' method on integer histograms currently allocates new memory
each time it is called.

This commit adds an optional *FloatHistogram parameter that can be used
to reuse span and bucket slices. It is up to the caller to make sure the
input float histogram is not used anymore after the call.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-08 10:22:59 +01:00
Björn Rabenstein 85078b968f
Merge pull request #13208 from linasm/panic-free-float-histogram-add-sub
FloatHistogram.Add/Sub: handle any schema change
2023-12-07 20:12:21 +01:00
Björn Rabenstein de1ec731c7
Merge pull request #13212 from linasm/optimize-floatBucketIterator-for-common-case
Histograms: optimize floatBucketIterator for common case
2023-12-07 20:11:58 +01:00
Linas Medziunas bd895baefc FloatHistogram.Add/Sub: handle any schema change
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-12-07 20:50:54 +02:00
Matthieu MOREL 9c4782f1cc
golangci-lint: enable testifylint linter (#13254)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-12-07 11:35:01 +00:00
Arve Knudsen 237bfea46b chunks.Reader: Fix typo in ChunkOrIterable doc string.
Also fix comment typo in `FloatHistogram.Sub`.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-12-07 08:28:45 +01:00
Oleksandr Redko 2a75604f8e
Enable default revive rules (#13068)
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-11-29 17:23:34 +00:00
Linas Medziunas 048886ae8a Histograms: optimize floatBucketIterator for common case
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-29 08:54:05 +02:00
Linas Medziunas 49613823f8 Reuse slices in [Float]Histogram.ReduceResolution
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-28 15:45:31 +02:00
Björn Rabenstein a6d4b8d97b
Merge pull request #13190 from linasm/fix-float-histogram-add-sub-mutating-argument
Fix FloatHistogram.Add/Sub mutating its argument
2023-11-28 14:04:50 +01:00
Linas Medziunas f99ecc376e Fix FloatHistogram.Add/Sub mutating its argument
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-26 09:26:34 +02:00
Linas Medziunas ea1862aab4 Explicit schema check in [Float]Histogram.ReduceResolution
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-25 11:38:15 +02:00
Bryan Boreham f0e1b592ab Scraping: use slices.sort for exemplars
The sort implementation using Go generics is used everywhere else
in Prometheus.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-24 14:42:26 +00:00
Bryan Boreham a3e02f35d6 labels: extract common code between slice and stringlabels
This reduces bulk and should avoid issues if a fix is made in one file
and not the other.

A few methods now call `Range()` instead of `range`, but nothing
performance-sensitive.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-17 18:20:03 +00:00
zenador 32ee1b15de
Fix error on ingesting out-of-order exemplars (#13021)
Fix and improve ingesting exemplars for native histograms.

See code comment for a detailed explanation of the algorithm.

Note that this changes the current behavior for all kind of samples slightly: We now allow exemplars with the same timestamp as during the last scrape if the value or the labels have changed.

Also note that we now do not ingest exemplars without timestamps for native histograms anymore.

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: Björn Rabenstein <github@rabenste.in>

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Signed-off-by: zenador <zenador@users.noreply.github.com>
Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
2023-11-16 15:07:37 +01:00
Bryan Boreham 1bfb3ed062
Labels: reduce allocations when creating from TSDB WAL (#13044)
* Labels: reduce allocations when creating from TSDB

When reading the WAL, by passing references into the buffer we can avoid
copying strings under `-tags stringlabels`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-14 11:36:35 +00:00
Ziqi Zhao e250f09b5d
change origin schema in ReduceResolution method of histogram and float histogram (#13116)
* change origin schema in ReduceResolution method of histogram and float histogram

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>

---------

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-11-10 14:33:34 +01:00
Ziqi Zhao ab2a7bb74f
add generic shrink function (#13001)
Add `ReduceResolution` method to `Histogram` and `FloatHistogram`

This takes the original `mergeToSchema` function and turns it into a more generic `reduceResolution` function, which is the building block for the new methods.

The methods will help with addressing #12864.

---------

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-11-08 14:43:05 +01:00
Matthieu MOREL 05fba53e57 web : use Go standard package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-08 06:37:59 +00:00
Julien Pivotto cf01ec2119
Merge pull request #13091 from mmorel-35/errorlint/util
util: use Go standard errors package
2023-11-07 21:07:25 -06:00
Linas Medziunas ebed7d0612 Change Validate to be a method on histogram structs
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-03 16:47:59 +02:00
Linas Medziunas 1f8aea11d6 Move histogram validation code to model/histogram
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-03 16:17:24 +02:00
Matthieu MOREL fe057fc60d use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-03 07:26:31 +00:00
Björn Rabenstein fae4561369
Merge pull request #12954 from linasm/linasm/optimize-floatBucketIterator2
Histogram performance: optimize floatBucketIterator
2023-11-02 23:59:32 +01:00
Oleksandr Redko fa90ca46e5 ci(lint): enable godot; append dot at the end of comments
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-10-31 19:53:38 +02:00
Linas Medžiūnas e3e22abadb
Merge branch 'main' into linasm/optimize-floatBucketIterator2
Signed-off-by: Linas Medžiūnas <linasm@users.noreply.github.com>
2023-10-24 16:27:10 +03:00
beorn7 b428416f06 textparse: Update comment about timestamp_ms protobuf parsing
By now, we know better what the plan is.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-10-19 17:54:42 +02:00
Arthur Silva Sens ef8e6ae780
Parse created timestamps from Prometheus Protobuf (#12973)
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
2023-10-18 19:04:02 +01:00
Marc Tuduri 1ce066e51c
More periods
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
2023-10-18 11:53:50 +02:00
Marc Tuduri af7c31ee10
PR feedback
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
2023-10-18 11:53:50 +02:00
Marc Tuduri 8fededf6ad
promql(histograms): Change sample total calculation for histograms
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
2023-10-18 11:51:11 +02:00
Björn Rabenstein f33bffa788
Merge pull request #12891 from linasm/fix-gaps-in-histogram-equals
Fix NaN checks in [Float]Histogram.Equals method
2023-10-18 00:35:17 +02:00
Linas Medziunas 62bbb81e29 Mention bucket values in the comment
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-10-14 21:30:40 +03:00
Linas Medziunas ec823d9daf Update comments, bitwise comparison of float buckets
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-10-09 16:09:46 +03:00
Linas Medziunas c5c5c569fa Histogram performance: optimize floatBucketIterator
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-10-09 09:40:59 +03:00
Matthieu MOREL 0a513f827d ci(lint): enable errorlint linter on model
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-09-29 21:11:33 +00:00
Julien Pivotto c52db2b196 Remove duplicate tests
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-09-29 10:41:13 +02:00
johncming 5d68ebb207 pkg/rulefmt: fix bug of validate.
Signed-off-by: johncming <johncming@yahoo.com>
2023-09-29 10:35:07 +02:00
Linas Medziunas 3c047a3518 Expand docs comments
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-09-28 09:08:09 +03:00
Linas Medziunas dfb6292600 Compare FloatHistogram.[Zero]Count float values as binary
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-09-28 09:06:54 +03:00
Linas Medziunas cbd01fc296 Fix NaN sum check in [Float]Histogram.Equals method
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-09-25 16:03:55 +03:00
Goutham Veeramachaneni 86729d4d7b
Update exp package (#12650) 2023-09-21 22:53:51 +02:00
Gregor Zeitlinger f01718262a
Unit tests for native histograms (#12668)
promql: Extend testing framework to support native histograms

This includes both the internal testing framework as well as the rules unit test feature of promtool.

This also adds a bunch of basic tests. Many of the code level tests can now be converted to tests within the framework, and more tests can be added easily.

---------

Signed-off-by: Harold Dost <h.dost@criteo.com>
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
Signed-off-by: Stephen Lang <stephen.lang@grafana.com>
Co-authored-by: Harold Dost <h.dost@criteo.com>
Co-authored-by: Stephen Lang <stephen.lang@grafana.com>
Co-authored-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
2023-08-25 23:35:42 +02:00
zenador 54aaa2bd7e
Add histogram_stdvar and histogram_stddev functions (#12614)
* Add new function: histogram_stdvar and histogram_stddev

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-08-24 21:02:14 +02:00
Björn Rabenstein 798c5737a0
Merge pull request #12711 from fatsheep9146/floathistogram-addsub-enhance
enhance float histogram add and sub method
2023-08-24 13:14:10 +02:00
Ziqi Zhao de172049ab fix lint error
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-08-24 07:27:33 +08:00
Ziqi Zhao d3633d4e76
Update model/histogram/float_histogram.go
Co-authored-by: Björn Rabenstein <github@rabenste.in>
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-08-24 07:17:23 +08:00