Commit graph

11585 commits

Author SHA1 Message Date
Oleg Zaytsev 4086a5f042 Merge branch 'main' into prometheus-2023-04-03-3923e83 2023-04-13 09:15:24 +02:00
Alex Le 01d0dda4fc Rename PopulateBlockFunc to BlockPopulator
Signed-off-by: Alex Le <leqiyue@amazon.com>
2023-04-12 14:18:20 -07:00
Łukasz Mierzwa b6573353c1 Add query_samples_total metric
query_samples_total is a counter that tracks the total number of samples loaded by all queries.

The goal with this metric is to be able to see the amount of 'work' done by Prometheus to service queries.
At the moment we have metrics with the number of queries, plus more detailed metrics showing how much time each step of a query takes.
While those metrics do help they don't show us the whole picture.
Queries that do load more samples are (in general) more expensive than queries that do load fewer samples.
This means that looking only at the number of queries doesn't tell us how much 'work' Prometheus received.
Adding a counter that tracks the total number of samples loaded allows us to see if there was a spike in the cost of queries, not just the number of them.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2023-04-12 14:05:06 +01:00
Björn Rabenstein 8ed90b567b
Merge pull request #12234 from aknuds1/chore/improve-histogram-comments
tsdb: Improve a couple of histogram documentation comments
2023-04-12 10:55:22 +02:00
Ben Ye fb67d368a2 use consistent error for instant and range query 400
Signed-off-by: Ben Ye <benye@amazon.com>
2023-04-11 13:45:34 -07:00
Björn Rabenstein 6e0a46900b
Merge pull request #12192 from leizor/leizor/prometheus/issues/11204
Add support for native histograms to concreteSeriesIterator
2023-04-11 12:30:35 +02:00
Arve Knudsen cca7178a12 tsdb: Improve a couple of histogram documentation comments
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-04-07 18:06:27 +02:00
Justin Lei f90013a5a0 Update storage/remote/codec.go
Co-authored-by: Björn Rabenstein <github@rabenste.in>
Signed-off-by: Justin Lei <97976793+leizor@users.noreply.github.com>
2023-04-06 09:54:15 -07:00
Justin Lei 83f43982c9 Add support for native histograms to concreteSeriesIterator
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-04-06 09:54:15 -07:00
Justin Lei 73ff91d182 Test fixes
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-04-06 09:42:59 -07:00
Justin Lei c770ba8047 Add comment linking to PR
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-04-06 09:19:32 -07:00
Justin Lei 79db04eb12 Adjust samplesPerChunk from 120 to 220
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-04-06 09:19:32 -07:00
Julien Pivotto 8dba9163f1
Merge pull request #12216 from prometheus/dependabot/go_modules/github.com/Azure/go-autorest/autorest/adal-0.9.23
build(deps): bump github.com/Azure/go-autorest/autorest/adal from 0.9.22 to 0.9.23
2023-04-05 10:24:09 +02:00
Julien Pivotto 391473141d
Check health & ready: move to flags (#12223)
This makes it more consistent with other command like import rules. We
don't have stricts rules and uniformity accross promtool unfortunately,
but I think it's better to only have the http config on relevant check
commands to avoid thinking Prometheus can e.g. check the config over the
wire.

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-04-05 09:45:39 +02:00
Soon-Ping 6cecb87941
Generalized rule group iteration evaluation hook (#11885)
Signed-off-by: Soon-Ping Phang <soonping@amazon.com>
2023-04-04 20:21:13 +02:00
Chris Marchbanks 044b3ecd14
Merge pull request #12203 from bboreham/raise-max-samples-per-send
remote-write: raise default samples per send to 2,000
2023-04-04 07:51:12 -06:00
Ganesh Vernekar e709b0b36e
Merge pull request #12127 from codesome/ooo-mmap-replay
Update OOO min/max time properly after replaying m-map chunks
2023-04-04 12:05:57 +05:30
Ganesh Vernekar 5588cab8b2
Merge pull request #12173 from bboreham/builder-no-empty-labels
labels: simplify call to get Labels from Builder
2023-04-04 12:02:55 +05:30
Alex Le 1936868e9d
Allow populate block logic in compact to be overriden outside Prometheus (#11711)
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <emoc1989@gmail.com>
2023-04-04 12:01:49 +05:30
dependabot[bot] 3923e83413
build(deps): bump bufbuild/buf-setup-action from 1.13.1 to 1.16.0 (#12209)
Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.13.1 to 1.16.0.
- [Release notes](https://github.com/bufbuild/buf-setup-action/releases)
- [Commits](https://github.com/bufbuild/buf-setup-action/compare/v1.13.1...v1.16.0)

---
updated-dependencies:
- dependency-name: bufbuild/buf-setup-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-03 23:25:07 +02:00
Nidhey Nitin Indurkar 3f7beeecc6
feat: health and readiness check of prometheus server in CLI (promtool) (#12096)
* feat: health and readiness check of prometheus server in CLI (promtool)

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>
2023-04-03 22:32:39 +02:00
dependabot[bot] c8b408cf5e
build(deps): bump github.com/Azure/go-autorest/autorest/adal
Bumps [github.com/Azure/go-autorest/autorest/adal](https://github.com/Azure/go-autorest) from 0.9.22 to 0.9.23.
- [Release notes](https://github.com/Azure/go-autorest/releases)
- [Changelog](https://github.com/Azure/go-autorest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Azure/go-autorest/compare/autorest/adal/v0.9.22...autorest/adal/v0.9.23)

---
updated-dependencies:
- dependency-name: github.com/Azure/go-autorest/autorest/adal
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 20:16:09 +00:00
Julien Pivotto 4af28f8cf6
Merge pull request #12204 from prymitive/cmd_tests
Use a random port in cmd/prometheus tests
2023-04-03 22:11:59 +02:00
Julien Pivotto be07be7ff1
Merge pull request #12208 from prometheus/dependabot/github_actions/actions/cache-3.3.1
build(deps): bump actions/cache from 3.2.4 to 3.3.1
2023-04-03 22:11:04 +02:00
Julien Pivotto b5249ba367
Merge pull request #12215 from prometheus/dependabot/go_modules/github.com/miekg/dns-1.1.53
build(deps): bump github.com/miekg/dns from 1.1.51 to 1.1.53
2023-04-03 22:10:42 +02:00
Julien Pivotto e37e53ac54
Merge pull request #12210 from prometheus/dependabot/github_actions/actions/setup-go-4
build(deps): bump actions/setup-go from 3 to 4
2023-04-03 22:10:14 +02:00
Julien Pivotto fa25c3b5e5
Merge pull request #12213 from prometheus/dependabot/go_modules/github.com/ionos-cloud/sdk-go/v6-6.1.5
build(deps): bump github.com/ionos-cloud/sdk-go/v6 from 6.1.4 to 6.1.5
2023-04-03 22:09:03 +02:00
Julien Pivotto 53c66ee391
Merge pull request #12211 from prometheus/dependabot/github_actions/prometheus/promci-0.1.0
build(deps): bump prometheus/promci from 0.0.2 to 0.1.0
2023-04-03 22:08:49 +02:00
Julien Pivotto bb6d7e36bd
Merge pull request #12214 from prometheus/dependabot/go_modules/github.com/digitalocean/godo-1.98.0
build(deps): bump github.com/digitalocean/godo from 1.97.0 to 1.98.0
2023-04-03 22:08:19 +02:00
Julien Pivotto b045f4169e
Merge pull request #12217 from prometheus/dependabot/go_modules/google.golang.org/api-0.114.0
build(deps): bump google.golang.org/api from 0.111.0 to 0.114.0
2023-04-03 22:08:07 +02:00
Julien Pivotto a65bcb22de
Merge pull request #12218 from prometheus/dependabot/go_modules/documentation/examples/remote_storage/github.com/prometheus/prometheus-0.43.0
build(deps): bump github.com/prometheus/prometheus from 0.42.0 to 0.43.0 in /documentation/examples/remote_storage
2023-04-03 22:07:57 +02:00
Bartlomiej Plotka b250479a65
Merge pull request #12220 from prometheus/superq/errcheck
Move errcheck excludes config
2023-04-03 19:09:12 +02:00
Ganesh Vernekar 291ab4d0bc
Add Jesús Vázquez as a TSDB maintainer (#12222)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-04-03 19:41:31 +05:30
Łukasz Mierzwa f2b9a39a48 Use a random port in cmd/prometheus tests
There are a few tests that will run prometheus command.
This can test if there's already something listening on port :9090 since --web.listen-address defaults to 0.0.0.0:9090.
To fix that we can tell prometheus to use a random port on loopback interface.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2023-04-03 11:21:50 +01:00
SuperQ 8b01189327
Move errcheck excludes config
Eliminate the need for a second config file for golangci-lint config
file by moving the list of errcheck exclude functions into the yaml
config.

Signed-off-by: SuperQ <superq@gmail.com>
2023-04-03 09:33:04 +02:00
Bryan Boreham e917202766 labels: make sure estimated size is not negative
Deleted labels are remembered, even if they were not in `base` or were
removed from `add`, so `base+add-del` could go negative.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-04-02 11:17:09 +01:00
Bryan Boreham 889ef998f4 remote-write: adjust MaxShards and Capacity for larger MaxSamplesPerSend
Keep the target throughput and total pending samples the same.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-04-02 11:00:39 +01:00
Hayk Davtyan 408f31f786
[WebUI/ScrapePoolList] Case-insensitive search of "Scrape Pools" (#12207)
Signed-off-by: hayk96 <hayko5999@gmail.com>
2023-04-02 11:37:58 +02:00
dependabot[bot] ce353727f1
build(deps): bump github.com/prometheus/prometheus
Bumps [github.com/prometheus/prometheus](https://github.com/prometheus/prometheus) from 0.42.0 to 0.43.0.
- [Release notes](https://github.com/prometheus/prometheus/releases)
- [Changelog](https://github.com/prometheus/prometheus/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/prometheus/compare/v0.42.0...v0.43.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/prometheus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-02 00:02:05 +00:00
dependabot[bot] 0b7cf5b368
build(deps): bump google.golang.org/api from 0.111.0 to 0.114.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.111.0 to 0.114.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.111.0...v0.114.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-02 00:00:56 +00:00
dependabot[bot] 6f2507656c
build(deps): bump github.com/miekg/dns from 1.1.51 to 1.1.53
Bumps [github.com/miekg/dns](https://github.com/miekg/dns) from 1.1.51 to 1.1.53.
- [Release notes](https://github.com/miekg/dns/releases)
- [Changelog](https://github.com/miekg/dns/blob/master/Makefile.release)
- [Commits](https://github.com/miekg/dns/compare/v1.1.51...v1.1.53)

---
updated-dependencies:
- dependency-name: github.com/miekg/dns
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-02 00:00:12 +00:00
dependabot[bot] bf988304d7
build(deps): bump github.com/digitalocean/godo from 1.97.0 to 1.98.0
Bumps [github.com/digitalocean/godo](https://github.com/digitalocean/godo) from 1.97.0 to 1.98.0.
- [Release notes](https://github.com/digitalocean/godo/releases)
- [Changelog](https://github.com/digitalocean/godo/blob/main/CHANGELOG.md)
- [Commits](https://github.com/digitalocean/godo/compare/v1.97.0...v1.98.0)

---
updated-dependencies:
- dependency-name: github.com/digitalocean/godo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-01 23:59:59 +00:00
dependabot[bot] babdbd8add
build(deps): bump github.com/ionos-cloud/sdk-go/v6 from 6.1.4 to 6.1.5
Bumps [github.com/ionos-cloud/sdk-go/v6](https://github.com/ionos-cloud/sdk-go) from 6.1.4 to 6.1.5.
- [Release notes](https://github.com/ionos-cloud/sdk-go/releases)
- [Changelog](https://github.com/ionos-cloud/sdk-go/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/ionos-cloud/sdk-go/compare/v6.1.4...v6.1.5)

---
updated-dependencies:
- dependency-name: github.com/ionos-cloud/sdk-go/v6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-01 23:59:48 +00:00
dependabot[bot] a9af885de2
build(deps): bump prometheus/promci from 0.0.2 to 0.1.0
Bumps [prometheus/promci](https://github.com/prometheus/promci) from 0.0.2 to 0.1.0.
- [Release notes](https://github.com/prometheus/promci/releases)
- [Commits](https://github.com/prometheus/promci/compare/v0.0.2...v0.1.0)

---
updated-dependencies:
- dependency-name: prometheus/promci
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-01 23:57:22 +00:00
dependabot[bot] e15b0a29ac
build(deps): bump actions/setup-go from 3 to 4
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-01 23:57:17 +00:00
dependabot[bot] 2bd2a6006d
build(deps): bump actions/cache from 3.2.4 to 3.3.1
Bumps [actions/cache](https://github.com/actions/cache) from 3.2.4 to 3.3.1.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3.2.4...v3.3.1)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-01 23:57:09 +00:00
Patrick Oyarzun ae170f644c
Optimize long alternate lists (#463)
* Use a prefix trie for long alternate lists

* Add test for non terminal node

* Fix panic in FuzzFastRegexMatcher_WithFuzzyRegularExpressions when the fuzzy regex is invalid

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Address PR feedback

* Update model/labels/regexp_test.go

Co-authored-by: Marco Pracucci <marco@pracucci.com>

* Replace trie with slice or map depending on input size

* Fix tests

* Pull in tests from @pracucci's branch

* Add setMatches back in

* Use stringMatcher when it's faster

* Fix linter

* Estimate alternates ahead of time

* Simplify construction with `IndexByte`

* Add test and early return for empty regexp.

* Fix race conditions in tests

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Marco Pracucci <marco@pracucci.com>
2023-04-01 08:35:35 +02:00
Bryan Boreham 2fc2e233a6 remote-write: raise default samples per send to 2,000
Larger messages cost less, because the per-message overheads at sender
and receiver are spread over more samples.

Example: scraping 1.5 million series every 15 seconds will send 50
messages per second.

Previous default was 500.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-03-31 08:47:06 +00:00
Marco Pracucci 9fe1bd2d63
Improve TestAnalyzeRealQueries (#464)
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-31 08:27:43 +00:00
Marco Pracucci 05a3a79015
Cache optimized regexp matchers (#465)
* Cache optimized regexp matchers

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Added BenchmarkNewFastRegexMatcher_CacheMisses

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Improved benchmark

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Improved benchmark

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Use LRU cache v2

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Run gofumpt

Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-03-31 04:05:26 +02:00