Commit graph

12444 commits

Author SHA1 Message Date
Bryan Boreham 925134e6de tsdb tests: make work with labels SymbolTable
Need to initialize decoders with SymbolTable.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 93b72ec5dd tsdb: create SymbolTables for labels as required
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 8c9b028536
Merge pull request #13641 from bboreham/pick-13633
Cut 2.50.1 with metadata api bugfix (cherry-picked #13633)
2024-02-26 11:31:35 +00:00
Bryan Boreham e537d6d1c1 Cut 2.50.1 with metadata api bugfix
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 10:00:21 +00:00
ismail simsek eecaa71ff1 Fix: metadata API using wrong field names (#13633)
Fix is to add json tags to `Metadata` struct. Absence of these tags
causes Go to use the field name, which starts with an upper-case
letter and breaks the protocol.

Extend tests to verify the JSON response.

Signed-off-by: ismail simsek <ismailsimsek09@gmail.com>
2024-02-26 09:54:39 +00:00
ismail simsek 2507469291
Fix: metadata API using wrong field names (#13633)
Fix is to add json tags to `Metadata` struct. Absence of these tags
causes Go to use the field name, which starts with an upper-case
letter and breaks the protocol.

Extend tests to verify the JSON response.

Signed-off-by: ismail simsek <ismailsimsek09@gmail.com>
2024-02-26 09:53:39 +00:00
Bryan Boreham 22890b1eb3 PromQL: improve warning for mixed values in aggregations
Aggregations discard the metric name, so don't try to
include it in the error message.

Add a test that generates this warning.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-25 19:57:06 +00:00
junya koyama 37613ef3b6
Merge branch 'chore/bump-to-go1.21' of https://github.com/arukiidou/prometheus into chore/bump-to-go1.21 2024-02-24 00:34:33 +09:00
junya koyama e171f6c8aa
go: fix go.mod version from 1.21.7 to 1.21
Signed-off-by: junya koyama <arukiidou@yahoo.co.jp>
2024-02-24 00:33:35 +09:00
Bryan Boreham 2ac1632eec storage/remote: improve symbol-table handling
On the incoming path, `writeHandler.write()` creates a new table for
each request.

`labelProtosToLabels` takes a `ScratchBuilder` now.

Call `NewScratchBuilder` as required in tests.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham 8f525b4ba4 storage/remote tests: refactor: extract function newTestQueueManager
To reduce repetition.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham b39286fd1f Add dedupelabels tag to not build regular labels
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham d51a5344cd labels: new version de-duplicating strings in SymbolTables
The individual strings for label names and values are held in a table,
and each Labels value is a run of varint-encoded indexes into that table.

When creating new labels, a sync.Mutex is locked around reads and writes.
When reading labels, there is no locking because the table of strings
used by those labels is immutable.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham 28191109a8 Labels: add fake versions of SymbolTable apis
So we can use them where necessary for internlabels implementation.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham d1af84f6ee Labels: move Builder and Reset out of common
New internstrings implementation is different.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham 5aa4473894 labels tests: extend TestBuilder
Start with empty base labels, also check new and re-used symbol tables

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham bb82a57e64 Labels: Call NewScratchBuilder in test_utils
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham cc5dc6a61b labels: use Equal instead of DeepEqual
This will work better with a different data structure.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham 4d6bb2e0e4
Merge pull request #13628 from bboreham/cleanup-13583
tsdb/wlog: small cleanup of WAL watcher after #13583
2024-02-23 12:39:17 +00:00
Bryan Boreham cc2c3f7c2b
Merge pull request #10782 from prymitive/gzip
Use github.com/klauspost/compress for gzip and zlib
2024-02-23 10:42:02 +00:00
Manik Rana 08832e2db8
histogram: add UI tests for rendering histogram (#13631)
* histogram: add tests for rendering histograms

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* histogram: add tests for rendering histograms

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: remove unused import

Signed-off-by: Manik Rana <manikrana54@gmail.com>

---------

Signed-off-by: Manik Rana <manikrana54@gmail.com>
2024-02-23 11:31:39 +01:00
George Krajcsovits c6c8f63516
Merge pull request #13607 from fionaliao/ooo-samples-appended-type
Add sample type label to outOfOrderSamplesAppended metric
2024-02-23 09:41:45 +01:00
Łukasz Mierzwa daeef6ffc3 Add lint checks for gzip/zlib
Since we want to use a faster gzip/zlib libraries we should add a lint check so that it's used everywhere

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-22 17:08:40 +00:00
Łukasz Mierzwa 5597020a60 Use github.com/klauspost/compress for gzip and zlib
klauspost/compress is a high quality drop-in replacement for common Go
compression libraries. Since Prometheus sends out a lot of HTTP requests
that often return compressed output having improved compression
libraries helps to save cpu & memory resources.
On a test Prometheus server I was able to see cpu reduction from 31 to
30 cores.

Benchmark results:

name                                old time/op    new time/op    delta
TargetScraperGzip/metrics=1-8         69.4µs ± 4%    69.2µs ± 3%     ~     (p=0.122 n=50+50)
TargetScraperGzip/metrics=100-8       84.3µs ± 2%    80.9µs ± 2%   -4.02%  (p=0.000 n=48+46)
TargetScraperGzip/metrics=1000-8       296µs ± 1%     274µs ±14%   -7.35%  (p=0.000 n=47+45)
TargetScraperGzip/metrics=10000-8     2.06ms ± 1%    1.66ms ± 2%  -19.34%  (p=0.000 n=47+45)
TargetScraperGzip/metrics=100000-8    20.9ms ± 2%    17.5ms ± 3%  -16.50%  (p=0.000 n=49+50)

name                                old alloc/op   new alloc/op   delta
TargetScraperGzip/metrics=1-8         6.06kB ± 0%    6.07kB ± 0%   +0.24%  (p=0.000 n=48+48)
TargetScraperGzip/metrics=100-8       7.04kB ± 0%    6.89kB ± 0%   -2.17%  (p=0.000 n=49+50)
TargetScraperGzip/metrics=1000-8      9.02kB ± 0%    8.35kB ± 1%   -7.49%  (p=0.000 n=50+50)
TargetScraperGzip/metrics=10000-8     18.1kB ± 1%    16.1kB ± 2%  -10.87%  (p=0.000 n=47+47)
TargetScraperGzip/metrics=100000-8    1.21MB ± 0%    1.01MB ± 2%  -16.69%  (p=0.000 n=36+50)

name                                old allocs/op  new allocs/op  delta
TargetScraperGzip/metrics=1-8           71.0 ± 0%      72.0 ± 0%   +1.41%  (p=0.000 n=50+50)
TargetScraperGzip/metrics=100-8         81.0 ± 0%      76.0 ± 0%   -6.17%  (p=0.000 n=50+50)
TargetScraperGzip/metrics=1000-8        92.0 ± 0%      83.0 ± 0%   -9.78%  (p=0.000 n=50+50)
TargetScraperGzip/metrics=10000-8       93.0 ± 0%      91.0 ± 0%   -2.15%  (p=0.000 n=50+50)
TargetScraperGzip/metrics=100000-8       111 ± 0%       135 ± 1%  +21.89%  (p=0.000 n=40+50)

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-22 17:08:15 +00:00
Łukasz Mierzwa 92e381b8a3 Add a scrape benchmark with gzipped responses
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-22 17:07:22 +00:00
Goutham Veeramachaneni bf2658bec1
Merge pull request #13627 from aknuds1/arve/sync-otlp
otlptranslator: Upgrade to v0.95.0
2024-02-22 22:34:18 +05:30
Augustin Husson 9738e48a7d
Merge pull request #13629 from prometheus/release-2.50
merge back release 2.50
2024-02-22 11:07:52 +01:00
Bryan Boreham 6ed56c9f04 WAL watcher: improve comments
Clarify in the first comment that it is `watch()` that waits, and reduce
verbiage.

The second comment was slightly contradictory to the first and otherwise
didn't seem to add much, since `currentSegment` was incremented just a
few lines later.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-22 09:32:46 +00:00
Augustin Husson 545c28d923
update v2.51 release date (#13606)
* update v2.51 release date

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* add Bryan as a next release shepherd

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

---------

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2024-02-22 10:23:26 +01:00
Augustin Husson 814b920e8a
Cut v2.50.0 (#13612)
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2024-02-22 10:21:59 +01:00
Bryan Boreham a975a83079 tsdb: clean up Watcher debug messages
Print lastSegment after it gets initialized.

Move variable declaration to first use.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-22 09:19:18 +00:00
Bryan Boreham 78f46bccca tsdb/wlog tests: remove unnecessary sleep check
Sleep() is documented to return immediately on negative or zero input.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-22 09:14:52 +00:00
Arve Knudsen bf5ca8cf38 otlptranslator: Upgrade to v0.95.0
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-02-22 09:12:07 +01:00
arukiidou d55fe5064d
Merge branch 'prometheus:main' into chore/bump-to-go1.21 2024-02-22 12:42:07 +09:00
Callum Styan 0c71230784
fix bug that would cause us to endlessly fall behind (#13583)
* fix bug that would cause us to only read from the WAL on the 15s
fallback timer if remote write had fallen behind and is no longer
reading from the WAL segment that is currently being written to

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* remove unintended logging, fix lint, plus allow test to take slightly
longer because cloud CI

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* address review feedback

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* fix watcher sleeps in test, flu brain is smooth

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* increase timeout, unfortunately cloud CI can require a longer timeout

Signed-off-by: Callum Styan <callumstyan@gmail.com>

---------

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2024-02-21 17:09:07 -08:00
Julien Pivotto e22b564049 Always align scrapes even if tolerance is bigger than 1% of scrape interval
When the scrape tolerance is bigger than 1% of the scrape interval, take
1% of the scrape interval as the tolerance instead of not aligning the
scrape at all.

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2024-02-21 15:09:21 +01:00
Bryan Boreham 14e48cd3ce Docs: update remote-write defaults
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-21 11:39:23 +00:00
Fiona Liao 841a133514 Move histogramsAppended to be more consistent
Signed-off-by: Fiona Liao <fiona.liao@grafana.com>
2024-02-21 11:15:04 +00:00
Fiona Liao 52389647b2 Add type label to outOfOrderSamplesAppended metric
Signed-off-by: Fiona Liao <fiona.liao@grafana.com>
2024-02-19 15:24:39 +00:00
Augustin Husson 93591ec5cc
Cut v2.50.0-rc.1 (#13602)
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2024-02-19 14:15:03 +01:00
junya koyama 39e858e524
go: bump go from 1.20 to 1.21
Signed-off-by: junya koyama <arukiidou@yahoo.co.jp>
2024-02-19 20:40:55 +09:00
junya koyama ebb91bc111
ci: bump Go from 1.21 to 1.22
Signed-off-by: junya koyama <arukiidou@yahoo.co.jp>
2024-02-19 20:40:00 +09:00
Bryan Boreham aba0071480
Merge pull request #13589 from bboreham/trace_id
Standardise exemplar label as "trace_id"
2024-02-19 09:34:04 +00:00
machine424 92544c00bf
discovery: kubernetes: Avoid creating unnecessary Kubernetes indexers in RoleEndpointSlice
This was due to a missing "return", see https://github.com/prometheus/prometheus/pull/13554#discussion_r1490965817

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-02-16 14:29:28 +01:00
Björn Rabenstein ac10cd4d99
Merge pull request #13271 from ywwg/owilliams/utf8
UTF-8: Add partial support for parsing UTF-8 metric and label names
2024-02-16 11:42:55 +01:00
Owen Williams a28d7865ad UTF-8: Add support for parsing UTF8 metric and label names
This adds support for the new grammar of `{"metric_name", "l1"="val"}` to promql and some of the exposition formats.
This grammar will also be valid for non-UTF-8 names.
UTF-8 names will not be considered valid unless model.NameValidationScheme is changed.

This does not update the go expfmt parser in text_parse.go, which will be addressed by https://github.com/prometheus/common/issues/554/.

Part of https://github.com/prometheus/prometheus/issues/13095

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-02-15 14:34:37 -05:00
Bryan Boreham c0e36e6bb3 Standardise exemplar label as "trace_id"
This is consistent with the OpenTelemetry standard, and an example in OpenMetrics.

https://github.com/open-telemetry/opentelemetry-specification/blob/89aa01348139/specification/metrics/data-model.md#exemplars
https://github.com/OpenObservability/OpenMetrics/blob/138654493130/specification/OpenMetrics.md#exemplars-1

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-15 14:20:08 +00:00
Bryan Boreham d595f5a9b1
Merge pull request #13585 from prometheus/metalmatze-bufbuild
Update `go get buf.build` in README.md
2024-02-15 14:08:18 +00:00
Matthias Loibl 0b38a5b774
Update go get buf.build in README.md
The old import path stopped working. 

```
go get go.buf.build/protocolbuffers/go/prometheus/prometheus
go: unrecognized import path "go.buf.build/protocolbuffers/go/prometheus/prometheus": https fetch: Get "https://go.buf.build/protocolbuffers/go/prometheus/prometheus?go-get=1": dial tcp: lookup go.buf.build on 192.168.2.1:53: no such host
```

We should instead tell users to use the new import paths. 

Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>
2024-02-15 12:12:36 +01:00
Darshan Chaudhary 0e8a2e44a4
Azure SD: check for nil before accessing and dereferencing (#13578)
Signed-off-by: darshanime <deathbullet@gmail.com>
2024-02-14 17:59:01 +01:00