Commit graph

12242 commits

Author SHA1 Message Date
Bryan Boreham 9051100aba Scraping: share buffer pool across all scrapes
Previously we had one per scrapePool, and one of those per configured
scraping job. Each pool holds a few unused buffers, so sharing one
across all scrapePools reduces total heap memory.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-23 10:23:34 +00:00
Bryan Boreham 34676a240e scrape: consistent function names for metadata
Too confusing to have `MetadataList` and `ListMetadata`, etc.
I standardised on the ones which are in an interface.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-23 09:08:02 +00:00
Charles Korn 2329fba0e5
Fix linting issues in comments (#13178)
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-11-23 09:07:23 +00:00
Bryan Boreham a2d5c02298
Merge pull request #13084 from charleskorn/concatenatingchunkiterator
Fix issue where `concatenatingChunkIterator` can obscure errors
2023-11-22 08:16:39 +00:00
Etourneau Gwenn b37258c99b
Added Caching of network interface for Azure (#12622)
* Added Caching of network interface for Azure

Signed-off-by: Etourneau Gwenn <getourneau@yugabyte.com>

* Rename Counter for Azure cache

Signed-off-by: Etourneau Gwenn <getourneau@yugabyte.com>

* Format with goimports

Signed-off-by: Etourneau Gwenn <getourneau@yugabyte.com>

* Updated duration comparaison
Enabled cache by default with 5x the default refresh time

Signed-off-by: Etourneau Gwenn <getourneau@yugabyte.com>

* Change random function

Signed-off-by: Etourneau Gwenn <getourneau@yugabyte.com>

* Remove refresh interval

Signed-off-by: Etourneau Gwenn <getourneau@yugabyte.com>

* Remove from config as well

Signed-off-by: Etourneau Gwenn <getourneau@yugabyte.com>

* Reformat config_test
Removed uneeded error

Signed-off-by: Etourneau Gwenn <getourneau@yugabyte.com>

---------

Signed-off-by: Etourneau Gwenn <getourneau@yugabyte.com>
2023-11-21 11:59:17 +00:00
Julien Pivotto aeb8de0f27
Merge pull request #13159 from bboreham/labels-common
labels: extract common code between slice and stringlabels
2023-11-21 10:38:45 +01:00
Ziqi Zhao 8fe9250f7d optimize the logic of break the loop of reducing resolution
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-11-21 16:56:56 +08:00
Bryan Boreham f095c33da1 scrape: simplify TargetsActive function
Since everything was serialized on a single mutex, it's exactly the same
if we process targets in sequence without starting goroutines.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-20 19:28:08 +00:00
Julien Pivotto aaf5d643b6
Merge pull request #13164 from manas-rust/main
fix: The automatically generated file is inconsistent with the file i…
2023-11-20 16:17:14 +01:00
Julien Pivotto 684018dfc6
Merge pull request #13166 from prymitive/disable_scrape_compression
Add enable_compression scrape config option
2023-11-20 16:16:48 +01:00
Bryan Boreham f13bc1a5c9
Merge pull request #13040 from bboreham/smaller-stripeseries
TSDB: make the global hash lookup table smaller
2023-11-20 12:12:09 +00:00
Łukasz Mierzwa 870627fbed Add enable_compression scrape config option
Currently Prometheus will always request gzip compression from the target when sending scrape requests.
HTTP compression does reduce the amount of bytes sent over the wire and so is often desirable.
The downside of compression is that it requires extra resources - cpu & memory.

This also affects the resource usage on the target since it has to compress the response
before sending it to Prometheus.

This change adds a new option to the scrape job configuration block: enable_compression.
The default is true so it remains the same as current Prometheus behaviour.

Setting this option to false allows users to disable compression between Prometheus
and the scraped target, which will require more bandwidth but it lowers the resource
usage of both Prometheus and the target.

Fixes #12319.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2023-11-20 12:02:55 +00:00
wangqing 7e2c6fc8f3 fix: The automatically generated file is inconsistent with the file in the code warehouse
reference: 3ef153b00c

Signed-off-by: wangqing <wangqing@uniontech.com>
2023-11-20 17:57:43 +08:00
Ben Kochie 9509ad082a
Merge pull request #12790 from prometheus/superq/bump-golangci-lint
Update golangci-lint.
2023-11-17 13:20:05 -06:00
Bryan Boreham a3e02f35d6 labels: extract common code between slice and stringlabels
This reduces bulk and should avoid issues if a fix is made in one file
and not the other.

A few methods now call `Range()` instead of `range`, but nothing
performance-sensitive.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-17 18:20:03 +00:00
Oleg Zaytsev f997c72f29
Make head block ULIDs descriptive (#13100)
* Make head block ULIDs descriptive

As far as I understand, these ULIDs aren't persisted anywhere, so it
should be safe to change them.

When debugging an issue, seeing an ULID like
`2ZBXFNYVVFDXFPGSB1CHFNYQTZ` or `33DXR7JA39CHDKMQ9C40H6YVVF` isn't very
helpful, so I propose to make them readable in their ULID string
version.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Set a different ULID for RangeHead

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

---------

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2023-11-17 12:29:36 +01:00
Julien Pivotto 1b84c01b76
Merge pull request #13143 from mmorel-35/patch-3
tsdb/tsdbutil: use Go standard errors
2023-11-17 10:23:04 +01:00
Julien Pivotto 9cb96ad2ea
Merge pull request #13142 from mmorel-35/patch-2
tsdb/fileutil: use Go standard errors
2023-11-17 10:20:40 +01:00
Julien Pivotto 58eca19ac0
Merge pull request #13141 from mmorel-35/patch-1
tsdb/errors: fix errorlint linter
2023-11-17 10:20:00 +01:00
zenador 32ee1b15de
Fix error on ingesting out-of-order exemplars (#13021)
Fix and improve ingesting exemplars for native histograms.

See code comment for a detailed explanation of the algorithm.

Note that this changes the current behavior for all kind of samples slightly: We now allow exemplars with the same timestamp as during the last scrape if the value or the labels have changed.

Also note that we now do not ingest exemplars without timestamps for native histograms anymore.

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: Björn Rabenstein <github@rabenste.in>

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Signed-off-by: zenador <zenador@users.noreply.github.com>
Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
2023-11-16 15:07:37 +01:00
Julien Pivotto 362a0b0d14
Merge pull request #13150 from kakkoyun/upgrade_golang_exp
go.*: Upgrade golang.org/x packages
2023-11-16 10:04:52 +01:00
Goutham Veeramachaneni d84d65ba8c
Merge pull request #13148 from gouthamve/bump-otelcol-to-0.88
Bump OTel Collector dependency to v0.88.0
2023-11-16 07:20:30 +01:00
Ziqi Zhao 10ebeb0a62 rename lastSum -> lastCount && enrich test case with expectBucketCount and expectSchema
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-11-16 13:00:11 +08:00
Levi Harrison 6d80b30990
Merge pull request #13151 from LeviHarrison/release-2.48.0
Release 2.48.0
2023-11-15 18:32:33 -05:00
Levi Harrison 41c7253333 Release 2.48.0
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-11-15 15:15:57 -05:00
Kemal Akkoyun b53254a818 Upgrade golang.org/x packages
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
2023-11-15 18:10:39 +01:00
Goutham 3048a88ae7
Add suffixes
Older version already did that. This upgrade needed manual opt-in

Signed-off-by: Goutham <gouthamve@gmail.com>
2023-11-15 15:52:18 +01:00
Goutham 4710679fbe
Skip golanglint-ci on copied folders
Signed-off-by: Goutham <gouthamve@gmail.com>
2023-11-15 15:30:09 +01:00
Goutham a99f48cc9f
Bump OTel Collector dependency to v0.88.0
I initially didn't copy the otlptranslator/prometheus folder because I
assumed it wouldn't get changes. But it did. So this PR fixes that and
updates the Collector version.

Supersedes: https://github.com/prometheus/prometheus/pull/12809

Signed-off-by: Goutham <gouthamve@gmail.com>
2023-11-15 15:18:14 +01:00
Matthieu MOREL d7c3bc4cb0
tsdb/tsdbutil: use Go standard errors
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-14 20:46:36 +01:00
Matthieu MOREL e60a508dd8 tsdb/errors: fix errorlint linter
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-14 19:16:12 +00:00
Matthieu MOREL e3041740e4 tsdb/fileutil: use Go standard errors
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-14 19:05:21 +00:00
Julien Pivotto cf528bef03
Merge pull request #13103 from roidelapluie/experimentalflag
Add feature flag for PromQL experimental functions.
2023-11-14 18:58:40 +01:00
Julien Pivotto c44cab45b9
Merge pull request #13135 from machine424/race-ww
remote-write: fix race condition between ApplyConfig and Notify
2023-11-14 18:00:37 +01:00
Julien Pivotto 13783cd1db
Merge pull request #13139 from mmorel-35/errorf
tsdb: replace errors.Errorf by fmt.Errorf
2023-11-14 18:00:23 +01:00
Julien Pivotto c92fbf3fdf Add feature flag for PromQL experimental functions.
This PR adds an Experimental flag to the functions.

This can be used by https://github.com/prometheus/prometheus/pull/13059
but also xrate and other future functions.

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-11-14 17:48:58 +01:00
Matthieu MOREL dd8871379a remplace errors.Errorf by fmt.Errorf
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-14 13:04:31 +00:00
Bryan Boreham 1bfb3ed062
Labels: reduce allocations when creating from TSDB WAL (#13044)
* Labels: reduce allocations when creating from TSDB

When reading the WAL, by passing references into the buffer we can avoid
copying strings under `-tags stringlabels`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-14 11:36:35 +00:00
machine424 413b713aa8
remote/storage.go: adjust Storage.Notify() to avoid a race condition with Storage.ApplyConfig()
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2023-11-14 10:07:45 +01:00
Julien Pivotto b718dca70b
Merge pull request #13108 from mmorel-35/patch-2
tsdb/chunkenc: use Go standard errors package
2023-11-14 00:54:35 +01:00
Julien Pivotto 3173feb5ef
Merge pull request #13107 from mmorel-35/patch-1
tsdb/agent: use Go standard errors package
2023-11-14 00:54:11 +01:00
Julien Pivotto 90ed7b08dc
Merge pull request #13124 from mmorel-35/patch-5
tsdb/index: use Go standard errors package
2023-11-14 00:53:49 +01:00
Julien Pivotto afc57eb306
Merge pull request #13109 from mmorel-35/patch-3
tsdb/chunks: use Go standard errors package
2023-11-14 00:52:59 +01:00
Julien Pivotto b6274ee747
Merge pull request #13114 from mmorel-35/patch-4
tsdb/wlog: use Go standard errors package
2023-11-14 00:52:21 +01:00
Julien Pivotto 7bcae56b6e
Merge pull request #13130 from mmorel-35/patch-6
tsdb/encoding: use Go standard errors package
2023-11-14 00:51:47 +01:00
Julien Pivotto f53af5c849
Merge pull request #13131 from mmorel-35/patch-7
tsdb/tombstones: use Go standard errors package
2023-11-14 00:51:31 +01:00
Julien Pivotto 17fe6bc296
Merge pull request #13133 from mmorel-35/patch-8
tsdb/record: use Go standard errors package
2023-11-14 00:51:16 +01:00
Bryan Boreham 41758972e4
web/api: optimize labelnames/values with 1 set of matchers (#12888)
* web/api: optimize labelnames/values with 1 set of matchers

If there is exactly one set of matchers provided, we can skip adding
the results to a map and getting them back out again.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

---------

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-13 19:28:48 +01:00
Bryan Boreham 65a443e6e3 TSDB: initialize conflicts map only when we need it.
Suggested by @songjiayang.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-13 16:20:31 +00:00
Bryan Boreham 37573d083b
Merge pull request #13056 from songjiayang/symbolCacheEntry-filed-alignment
TSDB block index writer: reduce memory used by symbol cache

Improve struct field alignment for symbolCacheEntry
2023-11-13 14:37:43 +00:00