* Append metadata to the WAL
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Remove extra whitespace; Reword some docstrings and comments
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Use RLock() for hasNewMetadata check
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Use single byte for metric type in RefMetadata
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Update proposed WAL format for single-byte type metadata
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Address first round of review comments
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Amend description of metadata in wal.md
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Correct key used to retrieve metadata from cache
When we're setting metadata entries in the scrapeCace, we're using the
p.Help(), p.Unit(), p.Type() helpers, which retrieve the series name and
use it as the cache key. When checking for cache entries though, we used
p.Series() as the key, which included the metric name _with_ its labels.
That meant that we were never actually hitting the cache. We're fixing
this by utiling the __name__ internal label for correctly getting the
cache entries after they've been set by setHelp(), setType() or
setUnit().
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Put feature behind a feature flag
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Reorder WAL format document
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Fix CR comments
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Extract logic about changing metadata in an anonymous function
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Implement new proposed WAL format and amend relevant tests
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Use 'const' for metadata field names
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Apply metadata to head memSeries in Commit, not in AppendMetadata
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Add docstring and rename extracted helper in scrape.go
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Fix review comments around TestMetadata* tests
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Rebase with merged TSDB changes; fix duplicate definitions after rebase
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Remove leftover changes on db_test.go
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Rename feature flag
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Simplify updateMetadata helper function
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
* Remove extra newline
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>
This fixes an occurrence of a loop variable being captured in a
parallel test (`TestInitialUpdate`). Prior to this commit, only the
last test case declared in that test would actually execute. To work
around this problem, we create a copy of the range variable before the
paralllel test, as suggested in the documentation for the `testing`
package:
https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks
The test immediately after the one fixed here (`TestInvalidFile`)
followed the same pattern but correctly created a copy of the loop
variable, illustrating how easy it is to forget to do this in
practice.
Issue was automatically found using the `loopvarcapture` linter.
Signed-off-by: Renato Costa <renato@cockroachlabs.com>
Signed-off-by: Renato Costa <renato@cockroachlabs.com>
* model/relabel: Add benchmark
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* model/relabel: re-use Builder across relabels
Saves memory allocations.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* labels.Builder: allow re-use of result slice
This reduces memory allocations where the caller has a suitable slice available.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* model/relabel: re-use source values slice
To reduce memory allocations.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Unwind one change causing test failures
Restore original behaviour in PopulateLabels, where we must not overwrite the input set.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* relabel: simplify values optimisation
Use a stack-based array for up to 16 source labels, which will be the
vast majority of cases.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* lint
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Metadata was added recently but doesn't seem to be used much, at least as far as I could identify.
Yet it's part of memSeries struct and so even when empty takes 48 bytes,
which is a lot given that without it memSeries requires 224 bytes.
This change turns it into a pointer on the struct, that get set only when metadata is actually set of given series.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
Some frameworks issue HEAD requests to determine health.
This resolvesprometheus/prometheus#11159
Signed-off-by: Nicolas Dumazet <nicdumz.commits@gmail.com>
Signed-off-by: Nicolas Dumazet <nicdumz.commits@gmail.com>
* Cut Prometheus v2.38.0-rc.0
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Include new docs PRs in the changelog
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Add "Scrape:" prefix to changelog entry
Co-authored-by: Levi Harrison <git@leviharrison.dev>
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Co-authored-by: Levi Harrison <git@leviharrison.dev>
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Co-authored-by: Levi Harrison <git@leviharrison.dev>
* Cut Prometheus v2.38.0-rc.0
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Include new docs PRs in the changelog
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Add "Scrape:" prefix to changelog entry
Co-authored-by: Levi Harrison <git@leviharrison.dev>
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Co-authored-by: Levi Harrison <git@leviharrison.dev>
We're loading the time window once per each Append() call, plus once in
the Commit(). While not extremely expensive, atomic operations are also
not cheap.
Additionally, it makes sense to keep the window consistent for a single
append.
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
OOOHeadIndexReader was using the headIndexReader PostingsForMatchers()
and LabelValues() implementation which lead to a very subtle bug that
led to wrong query results.
headIndexReader LabelValues() implementation checks if the query
timerange overlaps with the head maxt and mint and if it doesnt it
returns an empty list of values. Since this code was also used by the
ooo head it led to wrong results that we were not able to see in tests
because our queries where always from MinInt64 to MaxInt64. This commit
also adds a new test that performs multiple time range queries to make
sure this never happens again.
Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>
* Update Go dependencies ahead of v2.38
I skipped an update for the Azure client libraries because that was too
involved and needs some migration work from Azure SD maintainers.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Run "go mod tidy" again
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Currently, it's not explicitly called out which permissions are needed
for service discovery of EC2 instances. It's not super hard to figure
out, but it did involve a bit of guesswork when I did it yesterday, and
I figure it's worth saving people that effort.
I've also seen examples around the internet where people are granting
much broader permissions than they need to, so maybe we can save on that
by explicitly saying what's needed.
Signed-off-by: Chris Sinjakli <chris@sinjakli.co.uk>
