Commit graph

1084 commits

Author SHA1 Message Date
Arve Knudsen ef70a6e6cf Allocate tracing attributes up front
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-09 16:57:56 +01:00
Arve Knudsen 46bf26e841 Create Tracer once per PostingsForMatchersCache
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-09 15:31:31 +01:00
Arve Knudsen 403abaee79 Set span status on error
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-09 15:31:31 +01:00
Arve Knudsen 7c313be3b5 Remove unnecessary event attributes
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-09 15:31:31 +01:00
Arve Knudsen 2aabf7cc2c Improve tracing
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-09 15:31:31 +01:00
Arve Knudsen 25e2c05a47 Implement review feedback
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-09 15:31:31 +01:00
Arve Knudsen 6c0910a289 Update tsdb/postings_for_matchers_cache.go
Co-authored-by: Charles Korn <charleskorn@users.noreply.github.com>
2023-11-09 15:31:31 +01:00
Arve Knudsen ca4cdeb1af Apply suggestions from code review
Co-authored-by: Charles Korn <charleskorn@users.noreply.github.com>
2023-11-09 15:31:31 +01:00
Arve Knudsen 8ece24ddb0 PostingsForMatchersCache: Add tracing
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-11-09 15:31:31 +01:00
György Krajcsovits 1149f7e9e1 Fix lint errors: dot at comment end
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-11-02 21:09:38 +01:00
György Krajcsovits d42e296516 Merge remote-tracking branch 'upstream/main' into krajo/merge-upstream 2023-11-02 20:45:05 +01:00
beorn7 5dca994f64 Merge branch 'release-2.48' into beorn7/release 2023-11-02 19:58:33 +01:00
Jeanette Tan 52eb303031 Refactor assigning MinTime in histogram chunks
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 21:23:05 +08:00
Jeanette Tan 3ccaaa40ba Fix according to code review
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:37:07 +08:00
György Krajcsovits 4296ecbd14 tsdb/compact_test.go: test mixed typed series with PopulateBlock
Add testcase and update test so that it can test native histograms as well.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-11-02 13:33:42 +08:00
Jeanette Tan 27abf09e7f Fix missing MinTime in histogram chunks
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:39 +08:00
Jeanette Tan 2f7060bd5a Expand TestPopulateWithTombSeriesIterators to test earlier deletion intervals for histogram chunks as well as time-overlapping chunks
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:35 +08:00
Jeanette Tan 7a4a1127b7 Expand TestPopulateWithTombSeriesIterators to test min max times of chunks, including mixed chunks
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:33 +08:00
Jeanette Tan 04aabdd7cc Refactor TestPopulateWithDelSeriesIterator unit tests to reuse more code
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:30 +08:00
Jeanette Tan 46be85f2dc Make TestPopulateWithDelSeriesIterator tests cover histogram types and check MinTime
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-02 13:33:26 +08:00
Björn Rabenstein a43669e611
Merge pull request #12928 from alexandear/ci-enable-godot
ci(lint): enable godot; append dot at the end of comments
2023-11-01 17:15:41 +01:00
Dimitar Dimitrov 1cb47cd3f7
Attempt to remove unused revive linter directive 2023-11-01 14:25:51 +01:00
Dimitar Dimitrov dba5006d7a
Merge branch 'upstream/main' into dimitar/sync-with-upstream-main 2023-11-01 14:06:11 +01:00
Julien Pivotto f568221610
Merge pull request #13057 from prometheus/release-2.48
Merge release-2.48 back into main
2023-10-31 15:24:39 -04:00
Oleksandr Redko fa90ca46e5 ci(lint): enable godot; append dot at the end of comments
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-10-31 19:53:38 +02:00
Oleksandr Redko 8e5f0387a2
ci(lint): enable nolintlint and remove redundant comments (#12926)
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-10-31 12:35:13 +01:00
Jeanette Tan 6341ba7374 Merge remote-tracking branch 'upstream/main' into sync-upstream-20231026 2023-10-26 22:18:24 +08:00
zenador 80e977aae6
Remove NewPossibleNonCounterInfo and minimise creating empty annotations (#13012)
* Remove NewPossibleNonCounterInfo until it can be made more efficient, and avoid creating empty annotations as much as possible

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-10-24 17:36:07 +01:00
Arve Knudsen 6a4a87271b PostingsForMatchersCache test improvements
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-10-24 17:46:52 +02:00
Márcio Carôso dff1c395f6
Expose --storage.tsdb.retention.time in metric prometheus_tsdb_retention_limit_seconds (#12986)
* Expose --storage.tsdb.retention.time in a metric

Signed-off-by: Marcio Caroso <msscaroso@gmail.com>

---------

Signed-off-by: Marcio Caroso <msscaroso@gmail.com>
2023-10-24 13:34:42 +02:00
Björn Rabenstein 059f7f0738
Merge pull request #12997 from prometheus/wal-samples-size
TSDB: Pre-size buffer to read samples from WAL
2023-10-24 13:26:06 +02:00
Arve Knudsen efcd876b50 Ensure deterministic execution, for tests
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-10-20 16:21:26 +02:00
Charles Korn 6dcebc9e25
Don't allow cancelled contexts to poison the postings for matchers cache.
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-10-20 15:48:14 +11:00
Charles Korn 8df8db6c73
Add failing test
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-10-20 15:48:12 +11:00
Jeanette Tan 71a36d2396 Very minor refactor of the integer overflow fix
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-10-19 13:17:46 +08:00
Arve Knudsen a889bf6ad2 DB.UnorderedChunkQuerier: Remove unused ctx argument
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-10-18 18:23:32 +02:00
Jeanette Tan f898005c69 Merge remote-tracking branch 'upstream/main' into sync-upstream-20231018 2023-10-18 11:43:51 +08:00
Bryan Boreham 26fa2e8356 TSDB: Pre-size buffer to read samples from WAL
When reading the WAL this method is called with buffers from a pool, on
multiple goroutines. Pre-allocating sufficient size avoids slow growth
and many reallocations in `append`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-10-17 17:31:26 +00:00
George Krajcsovits 7d7b9eacff
Fix int32 overflow issues (#12978)
On a 32 bit architecture the size of int is 32 bits. Thus converting from
int64, uint64 can overflow it and flip the sign.

Try for yourself in playground:
package main

import "fmt"

func main() {
	x := int64(0x1F0000001)
	y := int64(1)
	z := int32(x - y) // numerically this is 0x1F0000000
	fmt.Printf("%v\n", z)
}

Prints -268435456 as if x was smaller.

Followup to #12650

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-10-16 16:23:26 +02:00
Paschalis Tsilias 42b8f2f5fc tsdb/agent: allow ingestion of OOO samples (#12897)
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Ganesh Vernekar 4df2f2432b Additionally wrap WBL replay error (#12406)
* Additionally wrap WBL replay error

Although WBL replay is already wrapped with errLoadWbl,
there are other errors that can happen during a WBL replay.
We should not try to repair WAL in those cases.

This commit additionally wraps the final error in Head.Init again
with errLoadWbl so that WBL replay errors can be identified properly.

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>
Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-10-15 13:47:42 -04:00
Paschalis Tsilias afab845e65
tsdb/agent: allow ingestion of OOO samples (#12897)
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
2023-10-13 16:33:09 +02:00
Ganesh Vernekar f5913266a1
Additionally wrap WBL replay error (#12406)
* Additionally wrap WBL replay error

Although WBL replay is already wrapped with errLoadWbl,
there are other errors that can happen during a WBL replay.
We should not try to repair WAL in those cases.

This commit additionally wraps the final error in Head.Init again
with errLoadWbl so that WBL replay errors can be identified properly.

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>
Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>
2023-10-13 14:21:35 +02:00
Arve Knudsen b5f0920506 Merge remote-tracking branch 'prometheus/main' into arve/sync-prometheus 2023-10-10 10:27:09 +02:00
Arve Knudsen 02680b42f6 tsdb: Avoid potential overflow in comparisons
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-10-10 10:11:33 +02:00
Arve Knudsen 26d07ee8d3 tsdb: Avoid potential overflow in SortFunc
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-10-10 09:49:10 +02:00
Oleg Zaytsev fe90dcccff
Revert ListPostings change (#12955)
Reverts change from https://github.com/prometheus/prometheus/pull/12906

The benchmarks show that it's slower when intersecting, which is a
common usage for ListPostings (when intersecting matchers from Head)

(old is before #12906, new is #12906):

                           │     old     │                 new                 │
                           │   sec/op    │   sec/op     vs base                │
Intersect/LongPostings1-16   20.54µ ± 1%   21.11µ ± 1%   +2.76% (p=0.000 n=20)
Intersect/LongPostings2-16   51.03m ± 1%   52.40m ± 2%   +2.69% (p=0.000 n=20)
Intersect/ManyPostings-16    194.2m ± 3%   332.1m ± 1%  +71.00% (p=0.000 n=20)
geomean                      5.882m        7.161m       +21.74%

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2023-10-09 17:25:18 +02:00
Oleg Zaytsev 5bd8c8c561
Clarify Postings.At() contract (#12921)
It's implicit, but should be explicit. It is invalid to call At() after
a failed call to Next() or Seek().

Following up on https://github.com/prometheus/prometheus/pull/12906

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2023-10-09 16:15:06 +02:00
Arve Knudsen 27d0e12f32
Update tsdb/compact.go
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2023-10-09 11:43:46 +02:00
Arve Knudsen a7737ab3ac Fix PostingsCloner tests
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-10-09 10:02:30 +02:00