Commit graph

12613 commits

Author SHA1 Message Date
Giedrius Statkevičius 3b8fe00767 tsdb/wlog: unregister metrics on WL close
Thanos can create and destroy TSDBs dynamically, and once a TSDB
disappears its files are deleted. Calculating the size of the
WAL then fails with errors like:

```
msg: "Failed to calculate size of "wal" dir", "err": "lstat
/tsdbdir/wal: no such file or directory", "caller": "wlog.go:271"
```

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
2024-04-11 11:30:05 +03:00
Julien 8b72ed77f8
Merge pull request #13869 from prometheus/dependabot/go_modules/go-opentelemetry-io-1393210b43
build(deps): bump the go-opentelemetry-io group with 3 updates
2024-04-05 14:59:36 +02:00
Julien 8eb9228af8
Merge pull request #13864 from yeya24/expose-compactor-metrics
Expose compactor metrics
2024-04-05 11:24:41 +02:00
Julien 48c8ec19bc
Merge pull request #13882 from prometheus/update-featureflag-docs
Update documentation about existing feature-flags
2024-04-05 11:22:46 +02:00
dandrucz 38b75bc0d7
Linode IPv6 Range support, Optional region filtering, Missing fields in Documentation fixed (#13774)
* Add support for discovering Linode IPv6 ranges associated with linodes.
* Add optional but recommended region filtering (faster queries, more relevant information).
* Added missing fields in configuration.md, updated linode test cases.
* Convert to TableDrivenTests as per tjhop request.

Signed-off-by: David Andruczyk <dandrucz@akamai.com>
2024-04-05 09:31:59 +01:00
Jonathan Halterman 113938aeb8
Log out of order when writing a block (#13888)
Signed-off-by: Jonathan Halterman <jonathan@grafana.com>
2024-04-04 14:26:13 +02:00
Bryan Boreham 8799581b24
Merge pull request #13554 from machine424/k8s-failures
discovery(k8s): add metric prometheus_sd_kubernetes_failures_total
2024-04-04 10:43:44 +01:00
Bryan Boreham 31491eb37c
Merge pull request #13885 from bboreham/readable-test
[TESTS] Truncate some long test names, for readability
2024-04-03 16:55:14 +01:00
Bryan Boreham 7c28521451 [TESTS] Truncate some long test names, for readability
The strings produced by these tests can run to thousands of characters,
which makes test logs difficult to read.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-03 10:10:39 +01:00
Charles Korn cd72ebb05f
promql: include more details in error message when creating test query fails or an unexpected series is returned (#13847)
* promql: include more details in error message when creating test query fails

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Include more details when an unexpected metric is returned

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-04-03 10:57:08 +02:00
Nicolas Takashi 8125634086
[refactor] moving mergedOOOChunks Iterator (#13881)
Signed-off-by: Nicolas Takashi <nicolas.tcs@hotmail.com>
2024-04-03 10:14:34 +02:00
Arthur Silva Sens db64d2dcdc
Update documentation about existing feature-flags
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
2024-04-02 19:18:57 -03:00
Julius Volz 9b7de47787
Remove unused Dmn field on EvalNodeHelper (#13877)
https://github.com/prometheus/prometheus/pull/13446 removed the last usage of
this field, but didn't remove the field.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2024-04-02 18:45:46 +02:00
dependabot[bot] 2e6c1c35a4
build(deps): bump the go-opentelemetry-io group with 3 updates
Bumps the go-opentelemetry-io group with 3 updates: [go.opentelemetry.io/collector/featuregate](https://github.com/open-telemetry/opentelemetry-collector), [go.opentelemetry.io/collector/pdata](https://github.com/open-telemetry/opentelemetry-collector) and [go.opentelemetry.io/collector/semconv](https://github.com/open-telemetry/opentelemetry-collector).


Updates `go.opentelemetry.io/collector/featuregate` from 1.3.0 to 1.4.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.3.0...pdata/v1.4.0)

Updates `go.opentelemetry.io/collector/pdata` from 1.3.0 to 1.4.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.3.0...pdata/v1.4.0)

Updates `go.opentelemetry.io/collector/semconv` from 0.96.0 to 0.97.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/v0.96.0...v0.97.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/collector/featuregate
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-opentelemetry-io
- dependency-name: go.opentelemetry.io/collector/pdata
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-opentelemetry-io
- dependency-name: go.opentelemetry.io/collector/semconv
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-opentelemetry-io
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-01 23:37:04 +00:00
Augustin Husson 9e2c335bab
Merge pull request #13855 from prometheus/merge-2.51-into-main
Merge 2.51.1 into main
2024-04-01 21:54:20 +02:00
carehabit a672662073
all: fix some typos (#13863)
Signed-off-by: carehabit <shenyuting@outlook.com>
2024-04-01 18:06:05 +02:00
Bryan Boreham 5e2caa32e7
Merge pull request #13856 from bboreham/release-ci-needs
CI: Publish step should require all Go tests to pass
2024-04-01 13:30:11 +01:00
Ben Ye ded35ef20d expose compactor metrics
Signed-off-by: Ben Ye <benye@amazon.com>
2024-03-31 15:10:29 -07:00
Arthur Silva Sens b51bbdd7ad
Merge pull request #13862 from nicolastakashi/refactor/moving-mergedOOOChunks-struct
[refactor] moving mergedOOOChunks to ooo_head_read
2024-03-29 20:51:38 -03:00
Nicolas Takashi 79d6750364
Merge branch 'prometheus:main' into refactor/moving-mergedOOOChunks-struct 2024-03-29 23:33:38 +00:00
Nicolas Takashi 0b762db154
[refactor] moving mergedOOOChunks to ooo_head_read
Signed-off-by: Nicolas Takashi <nicolas.tcs@hotmail.com>
2024-03-29 23:33:15 +00:00
Arthur Silva Sens 6a66f1f579
Merge pull request #13859 from prometheus/beorn7/release
Appoint release shepherds for v2.52 and v2.53
2024-03-28 19:09:57 -03:00
beorn7 fc3ad66539 Appoint release shepherds for v2.52 and v2.53
Note that we have delayed v2.52 by a week to avoid collisions with
events and travels.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-03-28 18:33:39 +01:00
Björn Rabenstein d81e41d58e
Merge pull request #13854 from prometheus/beorn7/testing
promql: Fix histogram comparison in test framework
2024-03-28 13:47:28 +01:00
Bryan Boreham 255098e053 CI: Publish step should require all Go tests to pass
This was an unintentional effect of splitting out Go tests into multiple
parallel blocks.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-03-28 10:06:28 +00:00
Bryan Boreham e1a5886c88
Merge branch 'main' into merge-2.51-into-main
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-03-28 10:00:25 +00:00
Bryan Boreham 855b5ac4b8
Cut release 2.51.1 (#13853)
* Cut release 2.51.1

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-03-28 09:23:45 +00:00
beorn7 65b4696b88 promql: Remove leftover debug output
Signed-off-by: beorn7 <beorn@grafana.com>
2024-03-27 19:02:27 +01:00
beorn7 2c1f9558b2 promql: Fix histogram comparison in test framework
The definition of histograms in the test framework may create
histograms in a non-compact form. Since histogram comparison relies on
exact equality of the bucket layout, we have to compact the histograms
created by the test framework language before comparing them to
histograms returned from the PromQL engine.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-03-27 19:00:16 +01:00
Bryan Boreham 2fc79839fd
Merge pull request #13851 from prometheus/krajo/pick-pr13846
Release 2.51: Cherry pick #13846 fix the bug of setting native histogram min bucket factor
2024-03-27 18:09:54 +01:00
Bryan Boreham ef7e9966d2
Merge pull request #13850 from prometheus/cherry-pick-13845
Release 2.51: Cherry-pick #13845 bugfix for DropMetricName
2024-03-27 18:06:44 +01:00
Björn Rabenstein b9a2a4e329
Merge pull request #13852 from prometheus/fix-hist-std-dev-var-negative
Fix hist std dev var negative
2024-03-27 17:58:03 +01:00
Jeanette Tan 4f2df329bd improve handling of empty buckets with infinite bounds in histogram std dev/var
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-03-27 17:06:12 +01:00
Jeanette Tan 22d0f4f114 improve handling of negative bounds in histogram std dev/var
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-03-27 17:06:12 +01:00
György Krajcsovits d64c6fe34f fix the bug of setting native histogram min bucket factor (#13846)
* fix the bug of setting native histogram min bucket factor

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>

* Add unit test for checking that min_bucket_factor is correctly applied

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-03-27 16:41:23 +01:00
Ziqi Zhao 64dfd8a158
fix the bug of setting native histogram min bucket factor (#13846)
* fix the bug of setting native histogram min bucket factor

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>

* Add unit test for checking that min_bucket_factor is correctly applied

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-03-27 16:32:37 +01:00
Domantas 3929d6500a [BUGFIX] labels: don't modify original labels in DropMetricName (#13845)
Restrict the capacity of first argument to `append()` to force an allocation.
This is for the slice implementation only.

Signed-off-by: Domantas Jadenkus <djadenkus@gmail.com>
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-03-27 15:32:03 +00:00
Bryan Boreham c6a42f8891
Merge pull request #13741 from bboreham/azure-test-labels
Azure Discovery tests: Add test for mapping VMs to labels
2024-03-27 11:36:17 +01:00
Domantas 435f330d0b
[BUGFIX] labels: don't modify original labels in DropMetricName (#13845)
Restrict the capacity of first argument to `append()` to force an allocation.
This is for the slice implementation only.

Signed-off-by: Domantas Jadenkus <djadenkus@gmail.com>
2024-03-27 10:35:17 +00:00
Björn Rabenstein 25a8d57671
Merge pull request #13844 from aknuds1/bugfix/wlog-checkpoint-float-histogram
tsdb/wlog.Checkpoint: Handle also float histograms
2024-03-26 17:41:43 +01:00
Arve Knudsen 35aab01de0 tsdb/wlog.Checkpoint: Handle also float histograms
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-03-26 15:27:32 +01:00
Bartlomiej Plotka de05d5e11b
Merge pull request #13776 from aknuds1/arve/checkpoint-histogram-samples
tsdb/wlog.Checkpoint: Fix counting of histogram samples
2024-03-26 12:33:48 +01:00
Bartlomiej Plotka 25578f2b22
[test] Merge pull request #13790 from aknuds1/arve/retention-commit
tsdb.BeyondTimeRetention: Fix comment and test at retention duration
2024-03-26 12:26:32 +01:00
Charles Korn 5cc97a1820
[tests]: extend test scripting language to support range queries (#13825)
* Extract method to make it easier to test.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Remove superfluous interface definition.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Add test cases for existing instant query functionality.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Add support for testing range queries

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Expand test coverage for instant queries and clarify error when a float is returned but a histogram is expected (or vice versa)

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Improve error message formatting

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Add test case for instant query command with invalid timestamp

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Fix linting warning.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Remove superfluous print statement and expected result

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Fix linting warning.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Add note about ordered range eval commands.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Check that matrix results are always sorted by labels.

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-03-26 11:22:22 +00:00
Nick Pillitteri 481f14e1c0
TSDB: Don't rely on integer overflow in head compaction check (#13755)
* TSDB: Don't compact the head block when empty

Don't compact the Head block if there have not yet been any samples
appended.

Previously, the logic for determining if the head should be compacted
relied on the default values for min and max time and integer overflow
when they were checked in `Head.compactable()`. The check in
`Head.compactable()` effectively did `math.MinInt64 - math.MaxInt64`
which overflowed and wrapped to `1`. Since `1` is less than `1.5`
times the chunk range, compaction did not happen. This was the correct
behavior but relying on overflow wrapping is surprising.

This change add a method for checking if the min and max time for the
head is unset and uses it to short-circuit compaction in that case.
It also replaces several explicit checks for the default value to
determine if the head has not yet had any samples added.

Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>
2024-03-26 12:17:38 +01:00
Bartlomiej Plotka d771cabc95
Merge pull request #13831 from prometheus/cherry-pick-13803
Release 2.51: Cherry-pick #13803 bugfix for label_join
2024-03-26 12:17:10 +01:00
Ben Ye ceca6c4716
[ENHANCEMENT] TSDB: Log more statistics during startup (#13838)
* log chunk snapshot and mmap chunks replay duration together with total replay duration

Signed-off-by: Ben Ye <benye@amazon.com>
2024-03-26 11:16:27 +00:00
Bryan Boreham 78c0fd2f4d
Merge pull request #13799 from machine424/wbl
chore(tsdb): set the wbl to nil as well in DBReadOnly.loadDataAsQuery…
2024-03-25 15:03:56 +01:00
Bryan Boreham f381ee3e55
Merge pull request #13834 from arturmelanchyk/lock-free-total
TSDB: lock-free total counter
2024-03-25 14:58:44 +01:00
Bryan Boreham 7d705ea9e8
Merge pull request #13830 from deterclosed/main
[DOCS] remove repeated word
2024-03-25 14:55:22 +01:00