Commit graph

12691 commits

Author SHA1 Message Date
Björn Rabenstein 11932cd345
Merge pull request #13637 from bboreham/agg-warning
PromQL: improve warning for mixed values in aggregations
2024-02-28 13:52:30 +01:00
Björn Rabenstein 4408f898bc
Merge pull request #13656 from prometheus/owilliams/quotetests
tests(utf8): confirm that other quote marks are handled correctly in promql
2024-02-27 23:16:43 +01:00
Owen Williams ac51a8024c tests(utf8): confirm that other quote marks are handled correctly in promql
Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-02-27 15:47:47 -05:00
Bryan Boreham befab52a14 Move v2.51 release date up
The previous release was forked on 25th Jan, so we should aim about 6 weeks after that.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-27 16:58:56 +00:00
Łukasz Mierzwa c013a3c1b5 Check of duplicated samples directly in scrapeCache.get()
Avoid extra map lookup by hooking check into cache get.

```
goos: linux
goarch: amd64
pkg: github.com/prometheus/prometheus/scrape
cpu: Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
                     │  main.txt   │              new.txt               │
                     │   sec/op    │   sec/op     vs base               │
ScrapeLoopAppend-8     66.72µ ± 0%   66.89µ ± 0%       ~ (p=0.879 n=50)
ScrapeLoopAppendOM-8   66.61µ ± 0%   66.89µ ± 1%       ~ (p=0.115 n=50)
geomean                66.66µ        66.89µ       +0.34%

                     │   main.txt    │               new.txt                │
                     │     B/op      │     B/op      vs base                │
ScrapeLoopAppend-8     20.17Ki ±  1%   20.12Ki ± 1%        ~ (p=0.343 n=50)
ScrapeLoopAppendOM-8   20.38Ki ± 10%   17.99Ki ± 2%  -11.69% (p=0.017 n=50)
geomean                20.27Ki         19.03Ki        -6.14%

                     │  main.txt  │               new.txt               │
                     │ allocs/op  │ allocs/op   vs base                 │
ScrapeLoopAppend-8     11.00 ± 0%   11.00 ± 0%       ~ (p=1.000 n=50) ¹
ScrapeLoopAppendOM-8   12.00 ± 0%   12.00 ± 0%       ~ (p=1.000 n=50) ¹
geomean                11.49        11.49       +0.00%
¹ all samples are equal
```

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-27 12:09:37 +00:00
Łukasz Mierzwa 21f8b35f5b Move staleness tracking out of checkAddError() calls
This call bloats checkAddError signature and logic, we can and should call it from the main scrape logic.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-27 11:36:16 +00:00
Łukasz Mierzwa 55dcaab41b Fix TestScrapeLoopDiscardDuplicateLabels test
This test calls Rollback() which is normally called from within append code.
Doing so means that staleness tracking data is outdated and need to by cycled manually.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-27 11:36:16 +00:00
Łukasz Mierzwa 50c81bed86 Check for duplicated series on a scrape
When Prometheus scrapes a target and it sees the same time series repeated multiple times it currently silently ignores that.
This change adds a test for that and fixes the scrape loop so that:

- Only first sample for each unique time series is appended
- Duplicated samples increment the prometheus_target_scrapes_sample_duplicate_timestamp_total metric

This allows one to identify such scrape jobs and targets.

Benchmark results:

```
name                            old time/op    new time/op    delta
ScrapeLoopAppend-8                64.8µs ± 2%    71.1µs ±20%   +9.75%  (p=0.000 n=10+10)
ScrapeLoopAppendOM-8              64.2µs ± 1%    68.5µs ± 7%   +6.71%  (p=0.000 n=9+10)
TargetsFromGroup/1_targets-8      14.2µs ± 1%    14.5µs ± 1%   +1.99%  (p=0.000 n=10+10)
TargetsFromGroup/10_targets-8      149µs ± 1%     152µs ± 1%   +2.05%  (p=0.000 n=9+10)
TargetsFromGroup/100_targets-8    1.49ms ± 4%    1.48ms ± 1%     ~     (p=0.796 n=10+10)

name                            old alloc/op   new alloc/op   delta
ScrapeLoopAppend-8                19.9kB ± 1%    17.8kB ± 3%  -10.23%  (p=0.000 n=8+10)
ScrapeLoopAppendOM-8              19.9kB ± 1%    18.3kB ±10%   -8.14%  (p=0.001 n=9+10)
TargetsFromGroup/1_targets-8      2.43kB ± 0%    2.43kB ± 0%   -0.15%  (p=0.045 n=10+10)
TargetsFromGroup/10_targets-8     24.3kB ± 0%    24.3kB ± 0%     ~     (p=0.083 n=10+9)
TargetsFromGroup/100_targets-8     243kB ± 0%     243kB ± 0%     ~     (p=0.720 n=9+10)

name                            old allocs/op  new allocs/op  delta
ScrapeLoopAppend-8                  9.00 ± 0%      9.00 ± 0%     ~     (all equal)
ScrapeLoopAppendOM-8                10.0 ± 0%      10.0 ± 0%     ~     (all equal)
TargetsFromGroup/1_targets-8        40.0 ± 0%      40.0 ± 0%     ~     (all equal)
TargetsFromGroup/10_targets-8        400 ± 0%       400 ± 0%     ~     (all equal)
TargetsFromGroup/100_targets-8     4.00k ± 0%     4.00k ± 0%     ~     (all equal)
```

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-27 11:36:16 +00:00
Łukasz Mierzwa 1a8ea78207 Fix BenchmarkScrapeLoopAppendOM
OpenMetrics requires EOF comment at the end of metrics body, but the makeTestMetrics() function doesn't append it.
This means this benchmark tests a response with errors but I don't think that was the intention.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-27 11:36:16 +00:00
Julien 0474b0bc39
Merge pull request #13621 from bboreham/rw-docs
Docs: update remote-write defaults
2024-02-27 12:28:16 +01:00
Julien 6ef810f5f8
Merge pull request #13650 from prometheus/dependabot/go_modules/github.com/docker/docker-25.0.3incompatible
build(deps): bump github.com/docker/docker from 25.0.0+incompatible to 25.0.3+incompatible
2024-02-27 12:18:17 +01:00
Julien 32a3b2e50d
Merge pull request #13648 from prometheus/dependabot/go_modules/k8s-io-a2ca04ec63
build(deps): bump the k8s-io group with 3 updates
2024-02-27 12:18:06 +01:00
Julien 8d1c3f8df5
Merge pull request #13649 from prometheus/dependabot/go_modules/github.com/aws/aws-sdk-go-1.50.25
build(deps): bump github.com/aws/aws-sdk-go from 1.50.0 to 1.50.25
2024-02-27 12:17:56 +01:00
Julien d753fd3fde
Merge pull request #13651 from prometheus/dependabot/go_modules/github.com/linode/linodego-1.29.0
build(deps): bump github.com/linode/linodego from 1.27.1 to 1.29.0
2024-02-27 12:17:46 +01:00
Björn Rabenstein dbb4055633
Merge pull request #13593 from machine424/no-index
discovery: kubernetes: Avoid creating unnecessary Kubernetes indexers…
2024-02-27 10:48:37 +01:00
Bryan Boreham 17edf98083
Merge pull request #13634 from bboreham/slim-parse-tests
refactor: metrics parsing: extract common checking code
2024-02-26 18:14:00 +00:00
gotjosh 0e25418eb6
Merge pull request #13652 from rnaveiras/rnaveiras-update-remote-write-docs-queue-default
[docs] Update remote write queue defaults
2024-02-26 17:38:00 +00:00
Bryan Boreham ba6aa95222 refactor: metrics parsing: extract common checking code
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 16:37:41 +00:00
Raúl Naveiras 58f528dcbc
[docs] Update remote write queue defaults
Update docs to reflect the default values in the
`config.DefaultQueueConfig`

Signed-off-by: Raúl Naveiras <me@raulnaveiras.com>
2024-02-26 15:59:29 +00:00
dependabot[bot] 851a28203c
build(deps): bump github.com/linode/linodego from 1.27.1 to 1.29.0
Bumps [github.com/linode/linodego](https://github.com/linode/linodego) from 1.27.1 to 1.29.0.
- [Release notes](https://github.com/linode/linodego/releases)
- [Commits](https://github.com/linode/linodego/compare/v1.27.1...v1.29.0)

---
updated-dependencies:
- dependency-name: github.com/linode/linodego
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-26 15:04:44 +00:00
dependabot[bot] 4a0541178d
build(deps): bump github.com/docker/docker
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 25.0.0+incompatible to 25.0.3+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v25.0.0...v25.0.3)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-26 15:04:22 +00:00
dependabot[bot] 67f13f3541
build(deps): bump github.com/aws/aws-sdk-go from 1.50.0 to 1.50.25
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.50.0 to 1.50.25.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.50.0...v1.50.25)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-26 15:03:53 +00:00
dependabot[bot] 3fc6577ec6
build(deps): bump the k8s-io group with 3 updates
Bumps the k8s-io group with 3 updates: [k8s.io/api](https://github.com/kubernetes/api), [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery) and [k8s.io/client-go](https://github.com/kubernetes/client-go).


Updates `k8s.io/api` from 0.28.6 to 0.29.2
- [Commits](https://github.com/kubernetes/api/compare/v0.28.6...v0.29.2)

Updates `k8s.io/apimachinery` from 0.28.6 to 0.29.2
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.28.6...v0.29.2)

Updates `k8s.io/client-go` from 0.28.6 to 0.29.2
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.28.6...v0.29.2)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: k8s-io
- dependency-name: k8s.io/apimachinery
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: k8s-io
- dependency-name: k8s.io/client-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: k8s-io
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-26 15:03:41 +00:00
dependabot[bot] 9fbfc9ea98
build(deps): bump actions/setup-go from 4.1.0 to 5.0.0
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4.1.0 to 5.0.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](93397bea11...0c52d547c9)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-26 15:03:04 +00:00
Julien e6d3d6abd9
Merge pull request #13604 from arukiidou/chore/bump-to-go1.21
ci: Update to Go version 1.22/ old to 1.21
2024-02-26 16:02:11 +01:00
Bryan Boreham d2817e9c91
Merge pull request #12304 from bboreham/labels-symboltable
Labels: reduce memory by de-duplicating strings in SymbolTables
2024-02-26 14:06:27 +00:00
Bryan Boreham 195a0f5ab8
remove uneccessary space
Co-authored-by: Julien <roidelapluie@o11y.eu>
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 14:02:06 +00:00
Julien 4284c29597
Merge pull request #13645 from bboreham/merge-2.50.1
Merge 2.50.1 into main
2024-02-26 14:10:43 +01:00
Bryan Boreham 518ab179fb Merge branch 'release-2.50' into merge-2.50.1 2024-02-26 11:50:06 +00:00
Bryan Boreham e1a741a0d7 labels: update copyright dates
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 55e7de04f8 model/labels (stringlabels): use strings.Clone
Suggestion from @colega.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham d16ce3c9bd model/labels (dedupelabels): small clarifications
Suggestion from @colega.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham c8c28efcb3 Run Go tests with -tags dedupelabels
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 0347148628 promql: fuzz test needs symbol table for parser
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 5f50d974c9 scraping: reset symbol table periodically
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 4e748b9cd8 scraping: re-use labels Builder in scrape report metrics
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham abb3a62f04 scraping: re-use symbol table for scrape loops
One symbol table for all loops in the same scrape pool, i.e. from the
same job.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 0403d098e1 scraping: re-use symbolTable for target discovery
Call labels.NewBuilderWithSymbolTable.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 9ba13de220 scraping: pass a Builder to get Target labels
This saves memory allocations from making a new Builder every time.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham eff3a13e19 model/textparse: parsers take a labels SymbolTable
This allows strings to be interned to save memory.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 123109e967 config: make work with SymbolTable labels
Need a SymbolTable when we expand ExternalLabels.

And we can't use labels.NewBuilder because we need to add blank values.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 3716326f3f rules: call NewScratchBuilder
Need to initialize ScratchBuilder with a SymbolTable.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 925134e6de tsdb tests: make work with labels SymbolTable
Need to initialize decoders with SymbolTable.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 93b72ec5dd tsdb: create SymbolTables for labels as required
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 8c9b028536
Merge pull request #13641 from bboreham/pick-13633
Cut 2.50.1 with metadata api bugfix (cherry-picked #13633)
2024-02-26 11:31:35 +00:00
Bryan Boreham e537d6d1c1 Cut 2.50.1 with metadata api bugfix
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 10:00:21 +00:00
ismail simsek eecaa71ff1 Fix: metadata API using wrong field names (#13633)
Fix is to add json tags to `Metadata` struct. Absence of these tags
causes Go to use the field name, which starts with an upper-case
letter and breaks the protocol.

Extend tests to verify the JSON response.

Signed-off-by: ismail simsek <ismailsimsek09@gmail.com>
2024-02-26 09:54:39 +00:00
ismail simsek 2507469291
Fix: metadata API using wrong field names (#13633)
Fix is to add json tags to `Metadata` struct. Absence of these tags
causes Go to use the field name, which starts with an upper-case
letter and breaks the protocol.

Extend tests to verify the JSON response.

Signed-off-by: ismail simsek <ismailsimsek09@gmail.com>
2024-02-26 09:53:39 +00:00
Bryan Boreham 22890b1eb3 PromQL: improve warning for mixed values in aggregations
Aggregations discard the metric name, so don't try to
include it in the error message.

Add a test that generates this warning.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-25 19:57:06 +00:00
junya koyama 37613ef3b6
Merge branch 'chore/bump-to-go1.21' of https://github.com/arukiidou/prometheus into chore/bump-to-go1.21 2024-02-24 00:34:33 +09:00