Commit graph

11091 commits

Author SHA1 Message Date
gotjosh e2a2790b2c
add more docs
Signed-off-by: gotjosh <josue.abreu@gmail.com>
2023-04-18 16:50:16 +01:00
gotjosh 96b6463f25
review comments
Signed-off-by: gotjosh <josue.abreu@gmail.com>
2023-04-18 16:26:32 +01:00
gotjosh f3394bf7a1
Rules API: Allow filtering by rule name
Introduces support for a new query parameter in the `/rules` API endpoint that allows filtering by rule names.

If all the rules of a group are filtered, we skip the group entirely.

Signed-off-by: gotjosh <josue.abreu@gmail.com>
2023-04-18 10:12:08 +01:00
Ganesh Vernekar 7309ac2721
Merge pull request #12257 from alexqyle/block-populator-rename
Rename PopulateBlockFunc to BlockPopulator
2023-04-14 13:35:01 +08:00
Björn Rabenstein 6b8573a846
Merge pull request #11687 from prometheus/beorn7/histogram
histograms: Optimize query performance
2023-04-13 20:14:09 +02:00
beorn7 717a3f8e25 storage: Manually expand genericAdd for specific types
This commit is doing what I would have expected that Go generics do
for me. However, the PromQL benchmarks show a significant runtime and
allocation increase with `genericAdd`, so this replaces it with
hand-coded non-generic versions of it.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-13 19:25:24 +02:00
beorn7 551de0346f promql: Do not return nil slices to the pool
Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-13 19:25:24 +02:00
beorn7 817a2396cb Name float values as "floats", not as "values"
In the past, every sample value was a float, so it was fine to call a
variable holding such a float "value" or "sample". With native
histograms, a sample might have a histogram value. And a histogram
value is still a value. Calling a float value just "value" or "sample"
or "V" is therefore misleading. Over the last few commits, I already
renamed many variables, but this cleans up a few more places where the
changes are more invasive.

Note that we do not to attempt naming in the JSON APIs or in the
protobufs. That would be quite a disruption. However, internally, we
can call variables as we want, and we should go with the option of
avoiding misunderstandings.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-13 19:25:24 +02:00
beorn7 462240bc78 storage: add specialized buffers to sampleRing
This utilizes the fact that most sampleRings will only contain samples
of one type. In this case, the generic interface is circumvented, and
a bespoke buffer for the one actually occurring sample type is
used. Should a sampleRing receive a sample of a different kind later,
it will transparently switch to the generic behavior.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-13 19:25:24 +02:00
beorn7 630bcb494b storage: Use separate sample types for histogram vs. float
Previously, we had one “polymorphous” `sample` type in the `storage`
package. This commit breaks it up into `fSample`, `hSample`, and
`fhSample`, each still implementing the `tsdbutil.Sample` interface.

This reduces allocations in `sampleRing.Add` but inflicts the penalty
of the interface wrapper, which makes things worse in total.

This commit therefore just demonstrates the step taken. The next
commit will tackle the interface overhead problem.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-13 19:25:24 +02:00
beorn7 c0879d64cf promql: Separate Point into FPoint and HPoint
In other words: Instead of having a “polymorphous” `Point` that can
either contain a float value or a histogram value, use an `FPoint` for
floats and an `HPoint` for histograms.

This seemingly small change has a _lot_ of repercussions throughout
the codebase.

The idea here is to avoid the increase in size of `Point` arrays that
happened after native histograms had been added.

The higher-level data structures (`Sample`, `Series`, etc.) are still
“polymorphous”. The same idea could be applied to them, but at each
step the trade-offs needed to be evaluated.

The idea with this change is to do the minimum necessary to get back
to pre-histogram performance for functions that do not touch
histograms. Here are comparisons for the `changes` function. The test
data doesn't include histograms yet. Ideally, there would be no change
in the benchmark result at all.

First runtime v2.39 compared to directly prior to this commit:

```
name                                                  old time/op    new time/op    delta
RangeQuery/expr=changes(a_one[1d]),steps=1-16            391µs ± 2%     542µs ± 1%  +38.58%  (p=0.000 n=9+8)
RangeQuery/expr=changes(a_one[1d]),steps=10-16           452µs ± 2%     617µs ± 2%  +36.48%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_one[1d]),steps=100-16         1.12ms ± 1%    1.36ms ± 2%  +21.58%  (p=0.000 n=8+10)
RangeQuery/expr=changes(a_one[1d]),steps=1000-16        7.83ms ± 1%    8.94ms ± 1%  +14.21%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_ten[1d]),steps=1-16           2.98ms ± 0%    3.30ms ± 1%  +10.67%  (p=0.000 n=9+10)
RangeQuery/expr=changes(a_ten[1d]),steps=10-16          3.66ms ± 1%    4.10ms ± 1%  +11.82%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_ten[1d]),steps=100-16         10.5ms ± 0%    11.8ms ± 1%  +12.50%  (p=0.000 n=8+10)
RangeQuery/expr=changes(a_ten[1d]),steps=1000-16        77.6ms ± 1%    87.4ms ± 1%  +12.63%  (p=0.000 n=9+9)
RangeQuery/expr=changes(a_hundred[1d]),steps=1-16       30.4ms ± 2%    32.8ms ± 1%   +8.01%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_hundred[1d]),steps=10-16      37.1ms ± 2%    40.6ms ± 2%   +9.64%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_hundred[1d]),steps=100-16      105ms ± 1%     117ms ± 1%  +11.69%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_hundred[1d]),steps=1000-16     783ms ± 3%     876ms ± 1%  +11.83%  (p=0.000 n=9+10)
```

And then runtime v2.39 compared to after this commit:

```
name                                                  old time/op    new time/op    delta
RangeQuery/expr=changes(a_one[1d]),steps=1-16            391µs ± 2%     547µs ± 1%  +39.84%  (p=0.000 n=9+8)
RangeQuery/expr=changes(a_one[1d]),steps=10-16           452µs ± 2%     616µs ± 2%  +36.15%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_one[1d]),steps=100-16         1.12ms ± 1%    1.26ms ± 1%  +12.20%  (p=0.000 n=8+10)
RangeQuery/expr=changes(a_one[1d]),steps=1000-16        7.83ms ± 1%    7.95ms ± 1%   +1.59%  (p=0.000 n=10+8)
RangeQuery/expr=changes(a_ten[1d]),steps=1-16           2.98ms ± 0%    3.38ms ± 2%  +13.49%  (p=0.000 n=9+10)
RangeQuery/expr=changes(a_ten[1d]),steps=10-16          3.66ms ± 1%    4.02ms ± 1%   +9.80%  (p=0.000 n=10+9)
RangeQuery/expr=changes(a_ten[1d]),steps=100-16         10.5ms ± 0%    10.8ms ± 1%   +3.08%  (p=0.000 n=8+10)
RangeQuery/expr=changes(a_ten[1d]),steps=1000-16        77.6ms ± 1%    78.1ms ± 1%   +0.58%  (p=0.035 n=9+10)
RangeQuery/expr=changes(a_hundred[1d]),steps=1-16       30.4ms ± 2%    33.5ms ± 4%  +10.18%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_hundred[1d]),steps=10-16      37.1ms ± 2%    40.0ms ± 1%   +7.98%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_hundred[1d]),steps=100-16      105ms ± 1%     107ms ± 1%   +1.92%  (p=0.000 n=10+10)
RangeQuery/expr=changes(a_hundred[1d]),steps=1000-16     783ms ± 3%     775ms ± 1%   -1.02%  (p=0.019 n=9+9)
```

In summary, the runtime doesn't really improve with this change for
queries with just a few steps. For queries with many steps, this
commit essentially reinstates the old performance. This is good
because the many-step queries are the one that matter most (longest
absolute runtime).

In terms of allocations, though, this commit doesn't make a dent at
all (numbers not shown). The reason is that most of the allocations
happen in the sampleRingIterator (in the storage package), which has
to be addressed in a separate commit.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-13 19:25:16 +02:00
Bartlomiej Plotka 136b48855a
Merge pull request #12259 from bboreham/labels-overwrite
labels: add ScratchBuilder.Overwrite for slice implementation
2023-04-13 14:07:04 +02:00
Bryan Boreham 10cc60af01 labels: add ScratchBuilder.Overwrite for slice implementation
This is a method used by some downstream projects; it was created to
optimize the implementation in `labels_string.go` but we should have one
for both implementations so the same code works with either.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-04-13 11:07:54 +00:00
Alex Le 01d0dda4fc Rename PopulateBlockFunc to BlockPopulator
Signed-off-by: Alex Le <leqiyue@amazon.com>
2023-04-12 14:18:20 -07:00
Björn Rabenstein 8ed90b567b
Merge pull request #12234 from aknuds1/chore/improve-histogram-comments
tsdb: Improve a couple of histogram documentation comments
2023-04-12 10:55:22 +02:00
Björn Rabenstein 6e0a46900b
Merge pull request #12192 from leizor/leizor/prometheus/issues/11204
Add support for native histograms to concreteSeriesIterator
2023-04-11 12:30:35 +02:00
Arve Knudsen cca7178a12 tsdb: Improve a couple of histogram documentation comments
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-04-07 18:06:27 +02:00
Justin Lei f90013a5a0 Update storage/remote/codec.go
Co-authored-by: Björn Rabenstein <github@rabenste.in>
Signed-off-by: Justin Lei <97976793+leizor@users.noreply.github.com>
2023-04-06 09:54:15 -07:00
Justin Lei 83f43982c9 Add support for native histograms to concreteSeriesIterator
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-04-06 09:54:15 -07:00
Julien Pivotto 8dba9163f1
Merge pull request #12216 from prometheus/dependabot/go_modules/github.com/Azure/go-autorest/autorest/adal-0.9.23
build(deps): bump github.com/Azure/go-autorest/autorest/adal from 0.9.22 to 0.9.23
2023-04-05 10:24:09 +02:00
Julien Pivotto 391473141d
Check health & ready: move to flags (#12223)
This makes it more consistent with other command like import rules. We
don't have stricts rules and uniformity accross promtool unfortunately,
but I think it's better to only have the http config on relevant check
commands to avoid thinking Prometheus can e.g. check the config over the
wire.

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-04-05 09:45:39 +02:00
Soon-Ping 6cecb87941
Generalized rule group iteration evaluation hook (#11885)
Signed-off-by: Soon-Ping Phang <soonping@amazon.com>
2023-04-04 20:21:13 +02:00
Chris Marchbanks 044b3ecd14
Merge pull request #12203 from bboreham/raise-max-samples-per-send
remote-write: raise default samples per send to 2,000
2023-04-04 07:51:12 -06:00
Ganesh Vernekar e709b0b36e
Merge pull request #12127 from codesome/ooo-mmap-replay
Update OOO min/max time properly after replaying m-map chunks
2023-04-04 12:05:57 +05:30
Ganesh Vernekar 5588cab8b2
Merge pull request #12173 from bboreham/builder-no-empty-labels
labels: simplify call to get Labels from Builder
2023-04-04 12:02:55 +05:30
Alex Le 1936868e9d
Allow populate block logic in compact to be overriden outside Prometheus (#11711)
Signed-off-by: Alex Le <leqiyue@amazon.com>
Signed-off-by: Alex Le <emoc1989@gmail.com>
2023-04-04 12:01:49 +05:30
dependabot[bot] 3923e83413
build(deps): bump bufbuild/buf-setup-action from 1.13.1 to 1.16.0 (#12209)
Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.13.1 to 1.16.0.
- [Release notes](https://github.com/bufbuild/buf-setup-action/releases)
- [Commits](https://github.com/bufbuild/buf-setup-action/compare/v1.13.1...v1.16.0)

---
updated-dependencies:
- dependency-name: bufbuild/buf-setup-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-03 23:25:07 +02:00
Nidhey Nitin Indurkar 3f7beeecc6
feat: health and readiness check of prometheus server in CLI (promtool) (#12096)
* feat: health and readiness check of prometheus server in CLI (promtool)

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>
2023-04-03 22:32:39 +02:00
dependabot[bot] c8b408cf5e
build(deps): bump github.com/Azure/go-autorest/autorest/adal
Bumps [github.com/Azure/go-autorest/autorest/adal](https://github.com/Azure/go-autorest) from 0.9.22 to 0.9.23.
- [Release notes](https://github.com/Azure/go-autorest/releases)
- [Changelog](https://github.com/Azure/go-autorest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Azure/go-autorest/compare/autorest/adal/v0.9.22...autorest/adal/v0.9.23)

---
updated-dependencies:
- dependency-name: github.com/Azure/go-autorest/autorest/adal
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 20:16:09 +00:00
Julien Pivotto 4af28f8cf6
Merge pull request #12204 from prymitive/cmd_tests
Use a random port in cmd/prometheus tests
2023-04-03 22:11:59 +02:00
Julien Pivotto be07be7ff1
Merge pull request #12208 from prometheus/dependabot/github_actions/actions/cache-3.3.1
build(deps): bump actions/cache from 3.2.4 to 3.3.1
2023-04-03 22:11:04 +02:00
Julien Pivotto b5249ba367
Merge pull request #12215 from prometheus/dependabot/go_modules/github.com/miekg/dns-1.1.53
build(deps): bump github.com/miekg/dns from 1.1.51 to 1.1.53
2023-04-03 22:10:42 +02:00
Julien Pivotto e37e53ac54
Merge pull request #12210 from prometheus/dependabot/github_actions/actions/setup-go-4
build(deps): bump actions/setup-go from 3 to 4
2023-04-03 22:10:14 +02:00
Julien Pivotto fa25c3b5e5
Merge pull request #12213 from prometheus/dependabot/go_modules/github.com/ionos-cloud/sdk-go/v6-6.1.5
build(deps): bump github.com/ionos-cloud/sdk-go/v6 from 6.1.4 to 6.1.5
2023-04-03 22:09:03 +02:00
Julien Pivotto 53c66ee391
Merge pull request #12211 from prometheus/dependabot/github_actions/prometheus/promci-0.1.0
build(deps): bump prometheus/promci from 0.0.2 to 0.1.0
2023-04-03 22:08:49 +02:00
Julien Pivotto bb6d7e36bd
Merge pull request #12214 from prometheus/dependabot/go_modules/github.com/digitalocean/godo-1.98.0
build(deps): bump github.com/digitalocean/godo from 1.97.0 to 1.98.0
2023-04-03 22:08:19 +02:00
Julien Pivotto b045f4169e
Merge pull request #12217 from prometheus/dependabot/go_modules/google.golang.org/api-0.114.0
build(deps): bump google.golang.org/api from 0.111.0 to 0.114.0
2023-04-03 22:08:07 +02:00
Julien Pivotto a65bcb22de
Merge pull request #12218 from prometheus/dependabot/go_modules/documentation/examples/remote_storage/github.com/prometheus/prometheus-0.43.0
build(deps): bump github.com/prometheus/prometheus from 0.42.0 to 0.43.0 in /documentation/examples/remote_storage
2023-04-03 22:07:57 +02:00
Bartlomiej Plotka b250479a65
Merge pull request #12220 from prometheus/superq/errcheck
Move errcheck excludes config
2023-04-03 19:09:12 +02:00
Ganesh Vernekar 291ab4d0bc
Add Jesús Vázquez as a TSDB maintainer (#12222)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-04-03 19:41:31 +05:30
Łukasz Mierzwa f2b9a39a48 Use a random port in cmd/prometheus tests
There are a few tests that will run prometheus command.
This can test if there's already something listening on port :9090 since --web.listen-address defaults to 0.0.0.0:9090.
To fix that we can tell prometheus to use a random port on loopback interface.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2023-04-03 11:21:50 +01:00
SuperQ 8b01189327
Move errcheck excludes config
Eliminate the need for a second config file for golangci-lint config
file by moving the list of errcheck exclude functions into the yaml
config.

Signed-off-by: SuperQ <superq@gmail.com>
2023-04-03 09:33:04 +02:00
Bryan Boreham e917202766 labels: make sure estimated size is not negative
Deleted labels are remembered, even if they were not in `base` or were
removed from `add`, so `base+add-del` could go negative.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-04-02 11:17:09 +01:00
Bryan Boreham 889ef998f4 remote-write: adjust MaxShards and Capacity for larger MaxSamplesPerSend
Keep the target throughput and total pending samples the same.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-04-02 11:00:39 +01:00
Hayk Davtyan 408f31f786
[WebUI/ScrapePoolList] Case-insensitive search of "Scrape Pools" (#12207)
Signed-off-by: hayk96 <hayko5999@gmail.com>
2023-04-02 11:37:58 +02:00
dependabot[bot] ce353727f1
build(deps): bump github.com/prometheus/prometheus
Bumps [github.com/prometheus/prometheus](https://github.com/prometheus/prometheus) from 0.42.0 to 0.43.0.
- [Release notes](https://github.com/prometheus/prometheus/releases)
- [Changelog](https://github.com/prometheus/prometheus/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/prometheus/compare/v0.42.0...v0.43.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/prometheus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-02 00:02:05 +00:00
dependabot[bot] 0b7cf5b368
build(deps): bump google.golang.org/api from 0.111.0 to 0.114.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.111.0 to 0.114.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.111.0...v0.114.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-02 00:00:56 +00:00
dependabot[bot] 6f2507656c
build(deps): bump github.com/miekg/dns from 1.1.51 to 1.1.53
Bumps [github.com/miekg/dns](https://github.com/miekg/dns) from 1.1.51 to 1.1.53.
- [Release notes](https://github.com/miekg/dns/releases)
- [Changelog](https://github.com/miekg/dns/blob/master/Makefile.release)
- [Commits](https://github.com/miekg/dns/compare/v1.1.51...v1.1.53)

---
updated-dependencies:
- dependency-name: github.com/miekg/dns
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-02 00:00:12 +00:00
dependabot[bot] bf988304d7
build(deps): bump github.com/digitalocean/godo from 1.97.0 to 1.98.0
Bumps [github.com/digitalocean/godo](https://github.com/digitalocean/godo) from 1.97.0 to 1.98.0.
- [Release notes](https://github.com/digitalocean/godo/releases)
- [Changelog](https://github.com/digitalocean/godo/blob/main/CHANGELOG.md)
- [Commits](https://github.com/digitalocean/godo/compare/v1.97.0...v1.98.0)

---
updated-dependencies:
- dependency-name: github.com/digitalocean/godo
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-01 23:59:59 +00:00
dependabot[bot] babdbd8add
build(deps): bump github.com/ionos-cloud/sdk-go/v6 from 6.1.4 to 6.1.5
Bumps [github.com/ionos-cloud/sdk-go/v6](https://github.com/ionos-cloud/sdk-go) from 6.1.4 to 6.1.5.
- [Release notes](https://github.com/ionos-cloud/sdk-go/releases)
- [Changelog](https://github.com/ionos-cloud/sdk-go/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/ionos-cloud/sdk-go/compare/v6.1.4...v6.1.5)

---
updated-dependencies:
- dependency-name: github.com/ionos-cloud/sdk-go/v6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-01 23:59:48 +00:00