Commit graph

11095 commits

Author SHA1 Message Date
Bryan Boreham 1aaabfee2d tsdb: make sharding function a parameter
Instead of relying on `labels.Hash()`, which may change, have the
caller pass in a shard function if required.

For most purposes `tsdb.Options.ShardFunc` is used, but the compactor
may be created independently so `NewLeveledCompactorWithChunkSize` also
takes a shard function parameter.

Regular Prometheus, which does not use block sharding, will have this
parameter as nil.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-01-12 11:41:22 +00:00
Bryan Boreham 2f58be840d service discovery: add config name to log messages
This makes it easier to connect a log message with the config it relates
to.

Each SD config has a name, either the scrape job name or something like
"config-0" for Alertmanager config.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-01-12 11:30:00 +00:00
Ganesh Vernekar d82ea2eb1c
Merge pull request #11838 from codesome/histo-rec
rules: Support native histograms
2023-01-12 12:35:15 +05:30
Ben Whetstone 32e9f6a39c Add container ID as a meta label for pod targets
Signed-off-by: Ben Whetstone <ben.whetstone@sysdig.com>
2023-01-11 11:44:36 -05:00
Björn Rabenstein ac96da3726
Merge pull request #11840 from prometheus/beorn7/histogram-gauge
tsdb: Add integer gauge histogram support
2023-01-11 15:10:51 +01:00
Ganesh Vernekar 98a0523e4a
rules: Test native histograms in recording rules
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-11 18:27:57 +05:30
beorn7 6dcd03dbf3 tsdb: Add integer gauge histogram support
This follows what #11783 has done for float gauge histograms.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-01-11 13:28:43 +01:00
beorn7 3e5ad99c33 textparse: Minor doc comment improvement
Signed-off-by: beorn7 <beorn@grafana.com>
2023-01-11 00:30:55 +01:00
Björn Rabenstein 6948fb1eb6
Merge pull request #11839 from prometheus/beorn7/histogram-gauge
textparse: Set CounterResetHint
2023-01-10 19:29:45 +01:00
beorn7 3f977e89ab textparse: Add tests for float and gauge histograms
Signed-off-by: beorn7 <beorn@grafana.com>
2023-01-10 18:42:05 +01:00
beorn7 c26891043f textparse: Set CounterResetHint
If a (float or integer) histogram is a gauge histogram, set the
CounterResetHint accordingly. (The default value is fine for the
normal counter histograms.)

Signed-off-by: beorn7 <beorn@grafana.com>
2023-01-10 16:25:23 +01:00
Peter Štibraný eaeda077ed
Stop unfinished async block writers if populateBlock encounters error (#393) 2023-01-10 15:54:20 +01:00
Björn Rabenstein f07adbd45f
Merge pull request #11814 from prometheus/beorn7/protobuf
textparse: Add gauge histogram support
2023-01-10 15:18:44 +01:00
Frederic Branczyk f007555be1
Merge pull request #11801 from ssbostan/main
update ServiceAccount for new kubernetes versions
2023-01-10 15:01:47 +01:00
Ganesh Vernekar 53a5071a72
rules: Support native histograms
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-10 19:07:24 +05:30
Ganesh Vernekar 57bcbf1888
Merge pull request #11783 from codesome/gauge-histogram
tsdb: Add gauge histogram support
2023-01-10 19:06:08 +05:30
Ganesh Vernekar 3c2ea91a83
tsdb: Test gauge float histograms
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-10 18:35:37 +05:30
Ganesh Vernekar 609b12d719
tsdb: Support gauge float histogram with recoding of chunk
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-10 17:48:09 +05:30
Ganesh Vernekar 8ad0d2d5d7
tsdb: Find union of two sets of histogram spans
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-10 17:43:33 +05:30
Ganesh Vernekar d7f5129042
tsdb: Add logic to determine appendable gauge float histograms
This is to check if a gauge histogram can be appended to the given chunk.
If not, it tells what changes to make to the chunk and the histogram
if possible.

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-10 17:43:33 +05:30
Ganesh Vernekar a87e7e9e33
tsdb: Add counter reset hint to histograms and support in WAL
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-10 17:41:53 +05:30
Oleg Zaytsev de93a279a0
Shortcut postings for matchers when empty postings are selected (#11813)
* Add more benchmark cases
* Add shortcuts for empty postings

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2023-01-10 15:21:49 +05:30
Ganesh Vernekar fd89d7892c
Merge pull request #11809 from bboreham/dont-sort-postings-values
tsdb: sort values for Postings only when required
2023-01-10 15:02:21 +05:30
Ganesh Vernekar c94a41c4b2
Merge pull request #11785 from Fish-pro/erroris
Use errors.Is to check for a specific error
2023-01-10 14:56:14 +05:30
Peter Štibraný 18bf6994fe Use err.
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2023-01-10 09:32:58 +01:00
Sam Jewell f88a0a7d83
Update example rules file to be valid with the default scrape config (#11692)
* Update docs example rules for default config

The prometheus download includes a default config to scrape itself.
This self-scraping prometheus doesn't include any metric named as
`http_inprogress_requests`, but does include one named
`prometheus_http_requests_total`.
Updating this example rule in the docs to one which can be used
out-of-the-box with the default download would be a nice improvement.

Signed-off-by: Sam Jewell <sam.jewell@grafana.com>

* Update syntax as per @LeviHarrison's review

Co-authored-by: Levi Harrison <levisamuelharrison@gmail.com>
Signed-off-by: Sam Jewell <2903904+samjewell@users.noreply.github.com>

Signed-off-by: Sam Jewell <sam.jewell@grafana.com>
Signed-off-by: Sam Jewell <2903904+samjewell@users.noreply.github.com>
Co-authored-by: Levi Harrison <levisamuelharrison@gmail.com>
2023-01-09 19:36:07 -05:00
Peter Štibraný fd284d4084 Fix bug in populateBlock when it could return with some asyncBlockWriter still running.
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2023-01-09 17:15:45 +01:00
George Krajcsovits f3d1f7756f
Merge pull request #391 from grafana/krajo/merge-jan08-from-upstream
merge jan08 from upstream
2023-01-09 15:20:16 +01:00
Julien Pivotto 48bccc50c8
Merge pull request #11831 from robbe-haesendonck/docs/fix-proxy-connect-header
docs: Changed occurences of proxy_connect_headers to proxy_connect_header
2023-01-09 14:19:26 +01:00
Robbe Haesendonck e802ddf435 docs: 📝 Changed occurences of proxy_connect_headers to proxy_connect_header
Since the struct defines proxy_connect_header instead of proxy_connect_headers, all relevant occurences of it were replaced with the correct configuration name as defined in the HTTPClientConfig struct.

Signed-off-by: Robbe Haesendonck <googleit@inuits.eu>
2023-01-09 14:11:00 +01:00
Levi Harrison 3b4cbf8da4
Inject readiness state through context (#11617)
Signed-off-by: Levi Harrison <git@leviharrison.dev>

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2023-01-09 00:04:00 +01:00
György Krajcsovits 069ceeaefb Fix linter errors after linter update
Updated linter golangci-lint from 1.45 to 1.50 previously

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-01-08 17:10:18 +01:00
Levi Harrison 5a485e15ea
Merge pull request #11822 from krajorama/krajo
Fix comment
2023-01-08 11:06:53 -05:00
György Krajcsovits 1fb0c8b688 Sync golangci-lint version with upstream
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-01-08 17:00:05 +01:00
György Krajcsovits f2bf0f0303 Fix gofumpt linter error
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-01-08 16:48:27 +01:00
György Krajcsovits a9ed61e91f Fix goftm linter error
How was this not inherited from upstream???

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-01-08 16:39:27 +01:00
György Krajcsovits 97626c9583 Fix comment
Comment was not updated when code changed from labels to builder
in #11717

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-01-08 16:29:02 +01:00
György Krajcsovits d524ab48b1 Fixes after merge
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-01-08 16:13:36 +01:00
György Krajcsovits 103c4fd289 Merge remote-tracking branch 'upstream/main' into main
# Conflicts:
#	.github/workflows/ci.yml
#	tsdb/block.go
#	tsdb/compact.go
#	tsdb/compact_test.go
#	tsdb/head_read.go
#	tsdb/index/index.go
#	tsdb/ooo_head_read.go
#	tsdb/querier_test.go
2023-01-08 14:55:44 +01:00
Ganesh Vernekar 57cab58c53
Merge pull request #11774 from codesome/rules-ooo
rules: Consider ErrTooOldSample in expected errors
2023-01-06 14:56:14 +05:30
Björn Rabenstein c49a28bb97
Merge pull request #11782 from codesome/floatappendabletest
tsdb: Improve TestFloatHistogramChunkAppendable and TestHistogramChunkAppendable
2023-01-05 17:15:10 +01:00
Saeid Bostandoust da6c014039
update ServiceAccount for new kubernetes versions
Signed-off-by: Saeid Bostandoust <49065072+ssbostan@users.noreply.github.com>
2023-01-05 15:43:48 +00:00
beorn7 b5d4a94e9d textparse: Add gauge histogram support to protobuf parsing
With this commit, the parser stops to see a gauge histogram (whether
native or conventional) as an unexpected metric type. It ingests it
normally, it even sets the `GaugeHistogram` type in the metadata (as
it has already done for a conventional gauge histogram scraped using
OpenMetrics), but it otherwise treats it as a normal counter-like
histogram.

Once #11783 is merged, though, it should be very easy to utilize the
type information.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-01-05 15:39:10 +01:00
beorn7 e9d9bb1b08 textparse: Handle unknown metric types in protobuf gracefully
So far, the parser hasn't validated that the type is valid in the
`Next()` call. Later, in the `Series()` call, however, it assumes that
we will only see valid types and therefore panics with `encountered
unexpected metric type, this is a bug`.

This commit fixes said bug by adding validation to the `Next()` call.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-01-05 15:21:18 +01:00
Bryan Boreham e61348d9f3 tsdb/index: fast-track postings for label=""
We need to special-case ""="" too, which is used in some tests to mean "everything".

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-01-05 14:05:54 +00:00
Bryan Boreham cf92cd2688 tsdb: sort values for Postings only when required
In the head and in v1 postings on disk, it makes no difference whether
postings are sorted. Only for v2 does the code step through in order.
So, move the sorting to where it is required, and thus skip it entirely
in the head.

Label values in on-disk blocks are already sorted, but `slices.Sort` is
very fast on already-sorted data so we don't bother checking.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-01-05 14:05:54 +00:00
Bryan Boreham 80ac0d7c82 promql: add benchmark for match against blank string
Blank strings are not handled efficiently by tsdb.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-01-05 14:05:54 +00:00
Ganesh Vernekar f1a332c496
rules: Consider ErrTooOldSample in expected errors
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-05 14:49:30 +05:30
Ganesh Vernekar 7ed1ddb338
tsdb: Improve TestHistogramChunkAppendable and add new cases
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-05 14:44:24 +05:30
Ganesh Vernekar fa0f04bbc6
Merge pull request #11805 from bboreham/fix-benchmark-intersect
tsdb/index: fix BenchmarkIntersect to do work on each loop
2023-01-04 18:19:14 +05:30