Commit graph

12109 commits

Author SHA1 Message Date
Matthieu MOREL e3041740e4 tsdb/fileutil: use Go standard errors
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-14 19:05:21 +00:00
Julien Pivotto cf528bef03
Merge pull request #13103 from roidelapluie/experimentalflag
Add feature flag for PromQL experimental functions.
2023-11-14 18:58:40 +01:00
Julien Pivotto c44cab45b9
Merge pull request #13135 from machine424/race-ww
remote-write: fix race condition between ApplyConfig and Notify
2023-11-14 18:00:37 +01:00
Julien Pivotto 13783cd1db
Merge pull request #13139 from mmorel-35/errorf
tsdb: replace errors.Errorf by fmt.Errorf
2023-11-14 18:00:23 +01:00
Julien Pivotto c92fbf3fdf Add feature flag for PromQL experimental functions.
This PR adds an Experimental flag to the functions.

This can be used by https://github.com/prometheus/prometheus/pull/13059
but also xrate and other future functions.

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-11-14 17:48:58 +01:00
Matthieu MOREL dd8871379a remplace errors.Errorf by fmt.Errorf
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-14 13:04:31 +00:00
Bryan Boreham 1bfb3ed062
Labels: reduce allocations when creating from TSDB WAL (#13044)
* Labels: reduce allocations when creating from TSDB

When reading the WAL, by passing references into the buffer we can avoid
copying strings under `-tags stringlabels`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-14 11:36:35 +00:00
machine424 413b713aa8
remote/storage.go: adjust Storage.Notify() to avoid a race condition with Storage.ApplyConfig()
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2023-11-14 10:07:45 +01:00
Julien Pivotto b718dca70b
Merge pull request #13108 from mmorel-35/patch-2
tsdb/chunkenc: use Go standard errors package
2023-11-14 00:54:35 +01:00
Julien Pivotto 3173feb5ef
Merge pull request #13107 from mmorel-35/patch-1
tsdb/agent: use Go standard errors package
2023-11-14 00:54:11 +01:00
Julien Pivotto 90ed7b08dc
Merge pull request #13124 from mmorel-35/patch-5
tsdb/index: use Go standard errors package
2023-11-14 00:53:49 +01:00
Julien Pivotto afc57eb306
Merge pull request #13109 from mmorel-35/patch-3
tsdb/chunks: use Go standard errors package
2023-11-14 00:52:59 +01:00
Julien Pivotto b6274ee747
Merge pull request #13114 from mmorel-35/patch-4
tsdb/wlog: use Go standard errors package
2023-11-14 00:52:21 +01:00
Julien Pivotto 7bcae56b6e
Merge pull request #13130 from mmorel-35/patch-6
tsdb/encoding: use Go standard errors package
2023-11-14 00:51:47 +01:00
Julien Pivotto f53af5c849
Merge pull request #13131 from mmorel-35/patch-7
tsdb/tombstones: use Go standard errors package
2023-11-14 00:51:31 +01:00
Julien Pivotto 17fe6bc296
Merge pull request #13133 from mmorel-35/patch-8
tsdb/record: use Go standard errors package
2023-11-14 00:51:16 +01:00
Bryan Boreham 41758972e4
web/api: optimize labelnames/values with 1 set of matchers (#12888)
* web/api: optimize labelnames/values with 1 set of matchers

If there is exactly one set of matchers provided, we can skip adding
the results to a map and getting them back out again.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

---------

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-13 19:28:48 +01:00
Bryan Boreham 65a443e6e3 TSDB: initialize conflicts map only when we need it.
Suggested by @songjiayang.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-13 16:20:31 +00:00
Bryan Boreham 37573d083b
Merge pull request #13056 from songjiayang/symbolCacheEntry-filed-alignment
TSDB block index writer: reduce memory used by symbol cache

Improve struct field alignment for symbolCacheEntry
2023-11-13 14:37:43 +00:00
Bryan Boreham 6613195487
Merge pull request #13097 from roidelapluie/gpu
Linode SD: Add __meta_linode_gpus label
2023-11-13 14:22:37 +00:00
machine424 08c17df244
remote/storage.go: add a test to highlight a race condition
between Storage.Notify() and Storage.ApplyConfig()

see https://github.com/prometheus/prometheus/issues/12747

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2023-11-13 13:23:53 +01:00
George Krajcsovits acc114fe55
Fix panic during tsdb Commit (#13092)
* Fix panic during tsdb Commit

Fixes the following

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x19deb45]

goroutine 651118930 [running]:
github.com/prometheus/prometheus/tsdb.(*headAppender).Commit(0xc19100f7c0)
	/drone/src/vendor/github.com/prometheus/prometheus/tsdb/head_append.go:855 +0x245
github.com/prometheus/prometheus/tsdb.dbAppender.Commit({{0x35bd6f0?, 0xc19100f7c0?}, 0xc000fa4c00?})
	/drone/src/vendor/github.com/prometheus/prometheus/tsdb/db.go:1159 +0x2f

We theorize that the panic happened due the the series referenced by the
exemplar being removed between AppendExemplar and Commit due to being idle.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-11-12 14:51:37 +00:00
Matthieu MOREL 469e415d09
Update record.go
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-11 21:01:24 +01:00
Matthieu MOREL 69c07ec6ae
Update record_test.go
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-11 20:57:42 +01:00
Matthieu MOREL 63691d82a5
tsdb/record: use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-11 20:52:49 +01:00
Matthieu MOREL c74b7ad4fb
Update tombstones.go
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-11 19:22:06 +01:00
Matthieu MOREL 118460a64f
tsdb/tombstones: use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-11 19:19:50 +01:00
Matthieu MOREL 4d6d3c1715
tsdb/encoding: use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-11 19:01:11 +01:00
George Krajcsovits 39a35d92bc
tsdb/head: wlog exemplars after samples (#13113)
When samples are committed in the head, they are also written to the WAL.
The order of WAL records should be sample then exemplar, but this was
not the case for native histogram samples. This PR fixes that.

The problem with the wrong order is that remote write reads the WAL and
sends the recorded timeseries in the WAL order, which means exemplars
arrived before histogram samples. If the receiving side is Prometheus
TSDB and the series has not existed before then the exemplar does not
currently create the series. Which means the exemplar is rejected and lost.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-11-11 17:30:16 +01:00
Ziqi Zhao b94f32f6fa automatically reduce resolution rather than fail scrape
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-11-11 22:24:47 +08:00
Ziqi Zhao e250f09b5d
change origin schema in ReduceResolution method of histogram and float histogram (#13116)
* change origin schema in ReduceResolution method of histogram and float histogram

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>

---------

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-11-10 14:33:34 +01:00
Jacob Baungård Hansen 80d2f992ae
codemirror-promql: Add request header to client (#13118)
With this commit we make it possible to adjust the request headers sent
to Prometheus by the codemirror-promql extension. This enables
customizing the headers sent, without re-implementing the Prometheus
client completely.

Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com>
2023-11-10 11:22:32 +01:00
Matthieu MOREL 2972cc5e8f tsdb/index: use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-09 21:37:41 +00:00
Julien Pivotto e8585e4d44
Merge pull request #13120 from roidelapluie/stalenesstracking
Follow-up to #13060: Add test to ensure staleness tracking
2023-11-09 10:48:16 -06:00
Julien Pivotto cfd83e71f7
Merge pull request #13110 from machine424/del-extra
head.go: Remove an unneeded snapshot trigger that was moved in https:…
2023-11-09 10:26:36 -06:00
Julien Pivotto 0fe34f6d78 Follow-up to #13060: Add test to ensure staleness tracking
This commit introduces an additional test in `scrape_test.go` to verify
staleness tracking when `trackTimestampStaleness` is enabled. The new
`TestScrapeLoopAppendStalenessIfTrackTimestampStaleness` function
asserts that the scrape loop correctly appends staleness markers when
necessary, reflecting the expected behavior with the feature flag turned
on.

The previous tests were only testing end of scrape staleness.

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-11-09 10:20:35 -06:00
Julien Pivotto 8b2b7d3d7a
Merge pull request #13117 from machine424/remote-headers
remote_write: add a unit test to make sure the write client sends the extra http headers as expected
2023-11-09 09:52:23 -06:00
machine424 0996b78326
remote_write: add a unit test to make sure the write client sends
the extra http headers as expected

This will help letting prometheus off the hook from situations like
https://github.com/prometheus/prometheus/issues/13030

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2023-11-09 15:56:48 +01:00
Bryan Boreham e6c0f69f98 TSDB: Only pay for hash collisions when they happen
Instead of a map of slices of `*memSeries`, ready for any of them to
hold series where hash values collide, split into a map of `*memSeries`
and a map of slices which is usually empty, since hash collisions are
a one-in-a-billion thing.

The `del` method gets more complicated, to maintain the invariant that
a series is only in one of the two maps.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-09 07:44:39 -06:00
Bryan Boreham ce4e757704 TSDB: refine variable naming in chunk gc
Slight further refactor.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-09 07:44:39 -06:00
Bryan Boreham 071d5732af TSDB: refactor cleanup of chunks and series
Extract the middle of the loop into a function, so it will be
easier to modify the `seriesHashmap` data structure.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-09 07:44:39 -06:00
machine424 a32fbc3658
head.go: Remove an unneeded snapshot trigger that was moved in https://github.com/prometheus/prometheus/pull/9328
and brougt back by mistake in 095f572d4a as part of https://github.com/prometheus/prometheus/pull/11447

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2023-11-09 11:46:46 +01:00
Matthieu MOREL fb48a351f0 tsdb/wlog: use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-08 21:41:58 +00:00
Nicolás Pazos 07e21cbba3
Remote Write 1.1: e2e benchmarks (#13102)
* Remote Write e2e benchmarks

Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>

* Prometheus ports automatically assigned

Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>

* make dashboard editable + more modular to different job label values

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Dashboard improvements

* memory stats
* diffs look at counter increases

Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>

* run script: absolute path for config templates

Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>

* grafana dashboard improvements

* show actual values of metrics
* add memory stats and diff

Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>

* dashboard changes

Signed-off-by: Callum Styan <callumstyan@gmail.com>

---------

Signed-off-by: Nicolás Pazos <npazosmendez@gmail.com>
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
2023-11-08 11:21:25 -08:00
songjiayang 443867f1aa symbolCacheEntry field type alignment, thus saving 8 bytes.
Signed-off-by: songjiayang <songjiayang1@gmail.com>
2023-11-09 00:43:27 +08:00
Ziqi Zhao ab2a7bb74f
add generic shrink function (#13001)
Add `ReduceResolution` method to `Histogram` and `FloatHistogram`

This takes the original `mergeToSchema` function and turns it into a more generic `reduceResolution` function, which is the building block for the new methods.

The methods will help with addressing #12864.

---------

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-11-08 14:43:05 +01:00
Arve Knudsen ae9221e152
tsdb/index.Symbols: Drop context argument from Lookup method (#13058)
Drop context argument from tsdb/index.Symbols.Lookup since lookup
should be fast and the context checking is a performance hit.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-08 13:08:33 +01:00
Matthieu MOREL ece8286305 tsdb/chunk: use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-08 09:19:44 +00:00
Matthieu MOREL b60f9f801e tsdb/chunkenc: use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-08 08:38:57 +00:00
Matthieu MOREL 724737006d
tsdb/agent: use Go standard errors package
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-11-08 09:22:31 +01:00