Commit graph

1167 commits

Author SHA1 Message Date
machine424 d1b4312f0a fix(wlog/watcher_test.go): make TestRun_AvoidNotifyWhenBehind more resilient
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-09-17 13:11:04 +02:00
Bryan Boreham d5f4fabd12
Merge pull request #14911 from bboreham/clarify-postings-benchmark
TSDB: Simplify benchmark regexps
2024-09-17 11:52:13 +02:00
Carrie Edwards 14e3c05ce8
tsdb: Add support for ingestion of out-of-order native histogram samples (#14546)
Add support for ingesting OOO native histograms

* Add flag for enabling and disabling OOO native histogram ingestion
* Update OOO querying tests to include native histogram samples
* Add OOO head tests
* Add test for OOO native histogram counter reset headers

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored by: Carrie Edwards <edwrdscarrie@gmail.com>
Co-authored by: Jeanette Tan <jeanette.tan@grafana.com>
Co-authored by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored by: Fiona Liao <fiona.liao@grafana.com>
2024-09-17 11:19:06 +02:00
Harry John 919dc0cbc6
storage: Update LabelQuerier interface to return sorted label values (#14849)
* Change LabelQuerier.LabelValues() to return sorted values

---------

Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>
2024-09-17 08:55:02 +02:00
Bryan Boreham a8133f3e87 TSDB: Simplify benchmark regexps
Several regexps were coded like `"^.*$"`, which is an unnatural
formulation nobody is likely to use. Inside `NewMatcher`, `^` and `$`
are added anyway, which makes the form in the benchmark redundant.

It even printed it out in the expected way.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-09-16 17:48:05 +01:00
George Krajcsovits 5aa3d8260a
TSDB: OOO native histograms: prep for multiple ooo head chunks (#14850)
Some checks failed
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
* tsdb: mmapCurrentOOOHeadChunk prepare for multiple ooo chunks

Currently float samples can only create a single ooo head chunk, but
native histograms can result in multiple due to counter resets, etc.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* tsdb: getOOOSeriesChunks prepare for multiple ooo chunks

Currently float samples can only create a single ooo head chunk, but
native histograms can result in multiple due to counter resets, etc.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-11 23:55:39 +01:00
Nathan Baulch 50cd453c8f
chore: Fix typos (#14868)
Some checks failed
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Push README to Docker Hub / Push README to Docker Hub (push) Has been cancelled
Push README to Docker Hub / Push README to quay.io (push) Has been cancelled
* Fix typos

---------

Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>
2024-09-10 22:32:03 +02:00
Bryan Boreham 16e5e99546
Merge pull request #14767 from bboreham/fix-encoding-comment
[Comment] Correct the comment on Decbuf.UvarintBytes
2024-09-09 12:52:36 +01:00
György Krajcsovits d3f4e7c223 Remove unnecessary conversion
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-09 12:51:02 +02:00
György Krajcsovits 60ab1cc5a5 BUGFIX: TSDB: panic in chunk querier
Followup to #14831

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-09 12:43:02 +02:00
George Krajcsovits 536d9f9ce9
BUGFIX: TSDB: panic in query during truncation with OOO head (#14831)
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Check if headQuerier is nil before trying to use it.

* TestQueryOOOHeadDuringTruncate: unit test to check query during truncate
Regression test for #14822

* Simulate race between query and Compact()

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-09-05 17:17:42 +01:00
Joshua Hesketh f2064c7987
NH: Do not re-use spans between histograms (#14771)
promql, tsdb (histograms): Do not re-use spans between histograms

When multiple points exist with the same native histogram schemas they
share their spans.
This causes a problem when a native histogram (NH) schema is modified (for example, during
a Sum) then the other NH's with the same spans are also modified. As such,
we should create a new Span for each NH. This will ensure NH's interfaces
are safe to use without considering the effect on other histograms.

At the moment this doesn't present itself as a problem because in all
aggregations and functions operating on native histograms they are copied
by the promql query engine first.

Signed-off-by: Joshua Hesketh <josh@nitrotech.org>

---------

Signed-off-by: Joshua Hesketh <josh@nitrotech.org>
2024-09-04 12:07:16 +02:00
George Krajcsovits 282fb1632a
Merge pull request #14772 from krajorama/fix-mockseriesiterator
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Fix: chunkenc.MockSeriesIterator
2024-09-03 16:55:26 +02:00
Arthur Silva Sens 442f24e099
chore: Simplify TestHeadAppender_AppendCTZeroSample (#14812)
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2024-09-02 21:30:37 +01:00
Arve Knudsen 2cfc7b244a
Merge pull request #14700 from shandongzhejiang/main
Comments: fix some function names
2024-09-02 18:59:28 +02:00
Oleg Zaytsev ce7d830f1f
Bring back BenchmarkLoadRealWLs (#14757)
This was part of #14525 which was reverted.
I still think that having this benchmark committed in to the repo is
useful.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-09-02 17:20:10 +01:00
György Krajcsovits a693dd19f2 Fix: chunkenc.MockSeriesIterator
Starts its index from 0 , but users call Next() before first sample
so it needs to start from -1

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-30 16:44:36 +02:00
Bryan Boreham 0a4f130b39 [Comment] Correct the comment on Decbuf.UvarintBytes
The value is valid when returned, but can become invalid later.

Return to previous wording.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-30 09:40:18 +01:00
Callum Styan a77f5007f9
fix bug with metadata for rw2 (#14766)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2024-08-30 08:14:20 +01:00
Bryan Boreham 1f38ae7bca [TESTS] TSDB: fix up OOO tests for new Series behaviour
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-29 10:59:09 +01:00
Bryan Boreham cde42f30e9 TSDB: streamline reading of overlapping head chunks
`getOOOSeriesChunks` was already finding sets of overlapping chunks; we
store those in a `multiMeta` struct so that `ChunkOrIterable` can
reconstruct an `Iterable` easily and predictably.

We no longer need a `MergeOOO` flag to indicate that this Meta should
be merged with other ones; this is explicit in the `multiMeta` structure.

We also no longer need `chunkMetaAndChunkDiskMapperRef`.

Add `wrapOOOHeadChunk` to defeat `chunkenc.Pool` - chunks are reset
during compaction, but if we wrap them (like `safeHeadChunk` was doing
then this is skipped) .

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-29 10:57:29 +01:00
Bryan Boreham 838e49e7b8 [REFACTOR] TSDB: move chunkFromSeries from headChunkReader to head
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-29 10:51:48 +01:00
Björn Rabenstein 1d6e0071b7
Merge pull request #14751 from riskrole/main
chore: fix some comments
2024-08-28 16:38:39 +02:00
riskrole 406bf775aa chore: fix some comments
Signed-off-by: riskrole <yuhang@before.tech>
2024-08-28 11:26:57 +08:00
Marco Pracucci ef649d5968
Revert " Store mmMaxTime in same field as seriesShard"
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2024-08-26 08:56:16 +02:00
György Krajcsovits 183bbc39a2 Make requesting merge with OOO head explicit in chunk.Meta
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-23 15:50:53 +02:00
György Krajcsovits 41c076196e New cases in Test_ChunkQuerier_OOOQuery and Test_Querier_OOOQuery
Case 1: OOO in-memory head chunk overlaps with first mmaped in-order chunk.

Query: |----------------------------------------------------------------|
InO:    |------mmap---------------||---------mem----------------------|
OOO:     |-----mem-----------|

This triggers ChunkOrIterableWithCopy not including OOO head chunks bug.

Similar to #14693 however testing the end of the interval doesn't
trigger the problem because there the in-order head chunk will be
trimmed with a tombstone, causing the code to switch to ChunkOrIterable
which was fixed.
See a36d1a8a92/tsdb/querier.go (L646)
where len(p.bufIter.Intervals) will be non zero, because it includes the
tombstone to trim the result to the query max time.

Thus a new test is added to check the overlap at the beginning of the
interval that has a separate chunk, which does not need trimming.

Note: same test doesn't fail for sample querier in Test_Querier_OOOQuery
as that doesn't use copy, that is copyHeadChunk is false in the if
condition above.

Case 2:

OOO mmaped head chunk overlaps with first mmaped in-order chunk.

Query: |----------------------------------------------------------------|
InO:    |------mmap---------------||---------mem----------------------|
OOO:     |-----mmap-----------|                             |--mem--|

In this case the meta contains the reference of the in-order chunk and
no indication that a merge is needed with the OOO mmaped chunk.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-23 15:50:47 +02:00
Arve Knudsen b0aba26ed5 tsdb: Fix ValNone typo in comment
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-08-23 08:20:20 +02:00
beorn7 0f760f63dd lint: Revamp our linting rules, mostly around doc comments
Several things done here:

- Set `max-issues-per-linter` to 0 so that we actually see all linter
  warnings and not just 50 per linter. (As we also set
  `max-same-issues` to 0, I assume this was the intention from the
  beginning.)

- Stop using the golangci-lint default excludes (by setting
  `exclude-use-default: false`. Those are too generous and don't match
  our style conventions. (I have re-added some of the excludes
  explicitly in this commit. See below.)

- Re-add the `errcheck` exclusion we have used so far via the
  defaults.

- Exclude the signature requirement `govet` has for `Seek` methods
  because we use non-standard `Seek` methods a lot. (But we keep other
  requirements, while the default excludes completely disabled the
  check for common method segnatures.)

- Exclude warnings about missing doc comments on exported symbols. (We
  used to be pretty adamant about doc comments, but stopped that at
  some point in the past. By now, we have about 500 missing doc
  comments. We may consider reintroducing this check, but that's
  outside of the scope of this commit. The default excludes of
  golangci-lint essentially ignore doc comments completely.)

- By stop using the default excludes, we now get warnings back on
  malformed doc comments. That's the most impactful change in this
  commit. It does not enforce doc comments (again), but _if_ there is
  a doc comment, it has to have the recommended form. (Most of the
  changes in this commit are fixing this form.)

- Improve wording/spelling of some comments in .golangci.yml, and
  remove an outdated comment.

- Leave `package-comments` inactive, but add a TODO asking if we
  should change that.

- Add a new sub-linter `comment-spacings` (and fix corresponding
  comments), which avoids missing spaces after the leading `//`.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-08-22 17:36:11 +02:00
Bryan Boreham 9a74d53935
[BUGFIX] TSDB: Fix query overlapping in-order and ooo head (#14693)
* tsdb: Unit test query overlapping in order and ooo head

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* TSDB: Merge overlapping head chunk

The basic idea is that getOOOSeriesChunks can populate Meta.Chunk, but since
it only returns one Meta per overlapping time-slot, that pointer may end up in a
Meta with a head-chunk ID. So we need HeadAndOOOChunkReader.ChunkOrIterable()
to call mergedChunks in that case.

Previously, mergedChunks was checking that meta.Ref was a valid OOO chunk reference,
but it never actually uses that reference; it just finds all chunks overlapping in time.
So we can delete that code.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-21 14:24:20 +01:00
shandongzhejiang b2712ff284 chore: fix some function names
Signed-off-by: shandongzhejiang <shandongzhejiang@icloud.com>
2024-08-21 11:09:37 +08:00
Arve Knudsen 3a78e76282 Upgrade golangci-lint to v1.60.1
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-08-18 12:13:25 +02:00
Bryan Boreham 87dccb1d1b
Merge pull request #14649 from machine424/ftest
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
fix(tsdb/db_test.go): close the corrupted chunk after creating it to satisfy Windows FS
2024-08-16 11:57:54 +01:00
Arve Knudsen 66388f706a
Merge pull request #14042 from aknuds1/arve/wlog-histograms
tsdb/wlog: Only treat unknown record types as failure
2024-08-16 12:00:49 +02:00
Björn Rabenstein 1daf7cdd62
Merge pull request #14626 from cuiweiyuan/main
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
chore: fix some function names
2024-08-15 11:46:21 +02:00
cuiweiyuan 1800af54f0 chore: fix some function names
Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>
2024-08-15 13:57:21 +08:00
Arve Knudsen b5d13a1ab5 Merge remote-tracking branch 'prometheus/main' into arve/wlog-histograms
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-08-14 19:04:53 +01:00
Bryan Boreham 512c67ec26 TSDB: Never go over maximum number of OOO chunks
In `mmapCurrentOOOHeadChunk`, check if the number is at the maximum and
drop the data with an error log. This is not expected to happen as the
maximum is over 8 million; that's 8 years of 1 sample every second.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 13:41:59 +01:00
Bryan Boreham 9135da1e4f TSDB: Review feedback
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Re-enable check in `createHeadWithOOOSamples` which wasn't really broken.
* Move code making `Block` into a `Queryable` into test file.
* Make `getSeriesChunks` return a slice (renamed `appendSeriesChunks`).
* Rename `oooMergedChunks` to `mergedChunks`.
* Improve comment on `ChunkOrIterableWithCopy`.
* Name return values from unpackHeadChunkRef.

Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com>
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 13:41:44 +01:00
Bryan Boreham 7ffd3ca280 TSDB: Cosmetic: move HeadAndOOO implementations where old code was
This makes the diffs easier to follow.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 13:41:13 +01:00
Bryan Boreham e95607b276 TSDB: Lock round access to labels, where necessary
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 13:41:13 +01:00
Bryan Boreham 26b3de0438 TSDB: Remove OOOHeadIndexReader
Use headIndexReader instead.

OOOCompactionHeadIndexReader needs to be expanded slightly, because it previously delegated to OOOHeadIndexReader.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 13:41:13 +01:00
Bryan Boreham a299c7b6d6 TSDB: Remove OOOHeadChunkReader
Use HeadAndOOOChunkReader instead.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 13:41:13 +01:00
Bryan Boreham e7e50a3afd TSDB: Remove code for querying OOO-head only
Just query via `HeadAndOOOQuerier`, which will skip series where no
in-order chunks are in range.

Now we don't need `OOORangeHead`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 13:41:13 +01:00
Bryan Boreham 0a2ff76881 TSDB tests: Fix up BenchmarkQueries
Was not working even on main.  Some cases still error.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 13:41:04 +01:00
Bryan Boreham f261597944 TSDB: Fix up LabelValues to work for OOO-only head
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 11:19:02 +01:00
Bryan Boreham 6529d6336c TSDB: NewHeadAndOOOChunkReader takes headChunkReader
So we can pass nil and have it read just OOO chunks.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 11:19:02 +01:00
Bryan Boreham e04d137649 [PERF] TSDB: Query head and ooo-head together
Add `HeadAndOOOQuerier` which iterates just once over series, then
where necessary merges chunks from in-order and out-of-order lists.

Add a ChunkQuerier for in-order and ooo together

Add copy-last-chunk behaviour to HeadAndOOOChunkReader

Out-of-order chunk IDs are distinguished from in-order by setting bit 23.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 11:19:02 +01:00
Bryan Boreham da31da3ea6 Refactor: extract selectSeriesSet and selectChunkSeriesSet
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 11:19:02 +01:00
Bryan Boreham 7e24844d08 Refactor: extract headChunkReader.chunkFromSeries()
For when you have a series locked already.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-08-14 11:19:02 +01:00