Commit graph

53 commits

Author SHA1 Message Date
Ben Kochie a06dcd7771
Update Makefile.common
* Bump promu to latest release.
* Bump golangci-lint to latest release.
* Add nolint to intentional error.

Signed-off-by: Ben Kochie <superq@gmail.com>
2021-04-13 09:33:45 +02:00
Julien Pivotto 6c56a1faaa
Testify: move to require (#8122)
* Testify: move to require

Moving testify to require to fail tests early in case of errors.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* More moves

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-29 09:43:23 +00:00
Julien Pivotto 1282d1b39c
Refactor test assertions (#8110)
* Refactor test assertions

This pull request gets rid of assert.True where possible to use
fine-grained assertions.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-27 11:06:53 +01:00
Julien Pivotto 4e5b1722b3
Move away from testutil, refactor imports (#8087)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-22 11:00:08 +02:00
Bartlomiej Plotka 9981b3f3ee
labels: Reduce allocated memory and latency of Hash method; Added tests. (#8025)
* labels: Reduce allocated memory by Hash method in edge cases; Added tests.

Old:
/tmp/___BenchmarkLabels_Hash_in_github_com_prometheus_prometheus_pkg_labels -test.v -test.bench ^\QBenchmarkLabels_Hash\E$ -test.run ^$
goos: linux
goarch: amd64
pkg: github.com/prometheus/prometheus/pkg/labels
BenchmarkLabels_Hash
BenchmarkLabels_Hash/typical_labels_under_1KB
BenchmarkLabels_Hash/typical_labels_under_1KB-12         	 5366161	       259 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabels_Hash/bigger_labels_over_1KB
BenchmarkLabels_Hash/bigger_labels_over_1KB-12           	 1700371	       767 ns/op	    2048 B/op	       1 allocs/op
BenchmarkLabels_Hash/extremely_large_label_value_10MB
BenchmarkLabels_Hash/extremely_large_label_value_10MB-12 	     356	   3743115 ns/op	10523442 B/op	       1 allocs/op
PASS

New:
/tmp/___BenchmarkLabels_Hash_in_github_com_prometheus_prometheus_pkg_labels -test.v -test.bench ^\QBenchmarkLabels_Hash\E$ -test.run ^$
goos: linux
goarch: amd64
pkg: github.com/prometheus/prometheus/pkg/labels
BenchmarkLabels_Hash
BenchmarkLabels_Hash/typical_labels_under_1KB
BenchmarkLabels_Hash/typical_labels_under_1KB-12         	 4758883	       259 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabels_Hash/bigger_labels_over_1KB
BenchmarkLabels_Hash/bigger_labels_over_1KB-12           	 3324492	       357 ns/op	      80 B/op	       1 allocs/op
BenchmarkLabels_Hash/extremely_large_label_value_10MB
BenchmarkLabels_Hash/extremely_large_label_value_10MB-12 	    1087	   1083949 ns/op	    9734 B/op	       1 allocs/op
PASS

Process finished with exit code 0

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed Kemal's comment.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Moved to v2 xxhash for improvements.

New:

/tmp/___BenchmarkLabels_Hash_in_github_com_prometheus_prometheus_pkg_labels -test.v -test.bench ^\QBenchmarkLabels_Hash\E$ -test.run ^$ -test.benchtime 10s
goos: linux
goarch: amd64
pkg: github.com/prometheus/prometheus/pkg/labels
BenchmarkLabels_Hash
BenchmarkLabels_Hash/typical_labels_under_1KB
BenchmarkLabels_Hash/typical_labels_under_1KB-12         	53447894	       221 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabels_Hash/bigger_labels_over_1KB
BenchmarkLabels_Hash/bigger_labels_over_1KB-12           	42341754	       326 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabels_Hash/extremely_large_label_value_10MB
BenchmarkLabels_Hash/extremely_large_label_value_10MB-12 	   10000	   1248546 ns/op	       0 B/op	       0 allocs/op
PASS

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed old xxhash package.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Pined minor version.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-10-15 11:31:28 +01:00
Bartlomiej Plotka dee6ac274e
labels: Extra tests and comment for WithoutLabels method. (#8024)
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-10-13 08:57:53 +01:00
Brian Brazil cf273f21bb
Add a testcase for #8040 (#8042)
This was already fixed by #8013, but add a test case anyway
in case the regexp engine changes in future.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-10-12 12:17:29 +01:00
Brian Brazil 924ce978b7
Don't do literal regex matching optimisation when case insensitive. (#8013)
Fixes #7994

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-10-06 13:16:26 +01:00
Guangwen Feng e6f2b8ad60
Fix golint issue in labels.go (#7730)
Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
2020-08-03 11:25:23 +01:00
Marco Pracucci 2f6bf7de4c
Optimise labels regex matchers containing a literal within the pattern (#7503)
* Added labels matchers regex fast path for literals within the regex

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2020-07-07 09:38:04 +01:00
Marco Pracucci cef4dd6fff
Optimized label regex matcher with literal prefix and/or suffix (#7453)
* Optimized label regex matcher with literal prefix and/or suffix

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Added license

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Added more tests cases with newlines

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Restored deleted test

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2020-06-26 15:19:09 +05:30
Chris Marchbanks ec45e3d029
Remove duplicate test in labels_test.go (#7461)
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2020-06-25 12:42:04 -06:00
Guangwen Feng b41adab735
Add unit test case for func FromStrings in labels.go (#7321)
Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
2020-06-24 11:54:30 +01:00
Hu Shuai e5ffa8c9a0
Add some unit tests for pkg/labels/labels.go (#7358)
This PR is about adding some unit tests for pkg/labels/labels.go.

Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-06-08 08:46:21 +01:00
Hu Shuai 6ff4814a49
Add some unit tests for pkg/labels/labels.go (#7347)
This PR is about adding some unit tests for pkg/labels/labels.go.

Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-06-05 09:48:33 +01:00
Callum Styan 5bb7f00d00
change labelset comparison in promql engine to avoid false positive during detection of duplicates (#7058)
* Use go1.14 new hash/maphash to hash both RHS and LHS instead of XOR'ing
which has been resulting in hash collisions.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Refactor engine labelset signature generation, just use labels.Labels
instead of hashes.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address review comments; function comments + store result of
lhs.String+rhs.String as key.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Replace all signatureFunc usage with signatureFuncString.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Make optimizations to labels String function and generation of rhs+lhs
as string in resultMetric.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Use separate string functions that don't use strconv just for engine
maps.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Use a byte invalid separator instead of quoting and have a buffer
attached to EvalNodeHelper instead of using a global pool in the labels
package.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address more review comments, labels has a function that now builds a
byte slice without turning it into a string.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Use two different non-ascii hex codes as byte separators between labels
and between sets of labels when building bytes of a Labels struct.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* We only need the 2nd byte invalid sep. at the beginning of a
labels.Bytes

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2020-05-12 14:03:15 -07:00
Hu Shuai da217cbde2
Add some unit tests for funcs in pkg/labels/labels.go (#7116)
This PR is about adding some unit tests for funcs in pkg/labels/labels.go.

Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-05-12 10:42:21 +01:00
ZouYu 06493b7034
add unit test TestLabels_String for pkg/labels/labels.go (#7150)
Signed-off-by: ZouYu <zouy.fnst@cn.fujitsu.com>
2020-04-22 12:32:47 +05:30
Hu Shuai 002b391dfc
Add unit test for Has in pkg/labels/labels.go (#7039)
This PR is about adding a unit test for Has in pkg/labels/labels.go.

Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-04-09 08:49:09 -07:00
Ben Kochie 269e7c8091
Fix golint issues.
Signed-off-by: Ben Kochie <superq@gmail.com>
2020-03-23 20:38:43 +01:00
Guangwen Feng a128fc3534
Add unit tests in labels.go (#6921)
* Add unit test for func Equal in labels.go
* Add unit test for func Compare in labels.go

Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
2020-03-03 14:17:54 +00:00
Guangwen Feng 87971fff90
Add unit test for func WithoutEmpty in labels.go (#6848)
Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
2020-02-19 11:56:12 +00:00
Guangwen Feng 14182c1420
Fix golint warning in labels.go (#6839)
Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
2020-02-19 07:25:44 +01:00
Ben Ye 492414542e
fix matcher for regex (#6540)
Signed-off-by: yeya24 <yb532204897@gmail.com>
2020-02-05 10:53:12 +00:00
Julien Pivotto 46d18112a3 tsdb: error on series with duplicate labels (#6664)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-20 11:05:27 +00:00
Hu Shuai a7f16749b7 Add unit test for Inverse in pkg/labels/matcher.go (#6586)
This PR is about adding a unit test for Inverse and updating the other test in pkg/labels/matcher.go.

Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-01-13 07:29:14 -07:00
Brian Brazil a9ed3b980d
Optimise labels.Compare (#6539)
strings.Compare isn't meant to be used, and this way we save one
comparison which is thus very slightly cheaper.

benchmark                                     old ns/op     new ns/op delta
BenchmarkPostingsForMatchers/Head/n="1"-4     236305440     233515705 -1.18%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-01-01 15:45:01 +00:00
Julius Volz ae93bae88f Fix: exclude metric name in labels.MatchLabels() (#6523)
This function is only used in one place to format an error message when
encountering multiple matches on the "one" side of a
one-to-many/many-to-one vector match, which is probably why nobodoy has
noticed this before. The hashing is already done correctly and excludes
the metric name label when using the "ignoring" matching modifier.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-12-27 09:32:19 +00:00
Tom Wilkie de0a772b8e Port tsdb to use pkg/labels. (#6326)
* Port tsdb to use pkg/labels.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Get tests passing.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Remove useless cast.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Appease linters.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-11-18 11:53:33 -08:00
Brian Brazil e62f30d497
Correctly handle empty labels from alert templates. (#5845)
Fixes https://github.com/prometheus/common/issues/36

Move logic handling this into the labels package,
so all the cases are handled in one place and we're
less likely to have this come up again.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-08-13 11:19:17 +01:00
Ganesh Vernekar 588eb20018
Efficient iteration and search in HashForLabels and HashWithoutLabels (#5707)
* Efficient iteration and search in Labels.HashForLabels

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Better names for variables

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* HashWithoutLabels optimizations

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor HashForLabels and HashWithoutLabels to take sorted names

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-06-28 18:22:51 +05:30
Thomas Jackson a000cec011 Re-use label builder in promql aggregation (#5641)
For my benchmarks on aggregation this reduces allocations by ~5% (~10%
time improvement):

```
benchmark                                                    old ns/op     new ns/op     delta
BenchmarkEvaluations/benchdata/aggregators.test/promxy-4     727692        649626        -10.73%

benchmark                                                    old allocs     new allocs     delta
BenchmarkEvaluations/benchdata/aggregators.test/promxy-4     2566           2434           -5.14%

benchmark                                                    old bytes     new bytes     delta
BenchmarkEvaluations/benchdata/aggregators.test/promxy-4     162760        148854        -8.54%
```

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>
2019-06-11 09:24:49 +01:00
Tom Wilkie c7b3535997 Use pkg/relabelling in remote write.
- Unmarshall external_labels config as labels.Labels, add tests.
- Convert some more uses of model.LabelSet to labels.Labels.
- Remove old relabel pkg (fixes #3647).
- Validate external label names.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-03-18 20:31:12 +00:00
Tariq Ibrahim a2a6e24f9f show list of offending labels in the error message in many-to-many scenarios (#5189)
Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-02-09 10:17:52 +01:00
JoeWrightss e158c53fa9 Fix some typos in comment (#5175)
Signed-off-by: zhoulin xie <zhoulin.xie@daocloud.io>
2019-02-01 14:35:32 +00:00
Alin Sinpalean 96fb0b2155 Optimize PromQL aggregations (#4248)
* Compute hash of label subsets without creating a LabelSet first.

Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>
2018-07-18 04:56:27 +01:00
Krasi Georgiev 675ce533c9 refactored TestScrapeLoopAppend and added a test for empty labels 2018-02-20 11:05:54 +00:00
Brian Brazil 4addee2bee Fix honor_labels for empty labels, prune empty labels.
The semantics of honor_labels are that if a target exposes
and empty label it will override the target labels. This PR
fixes that by once again distinguishing between empty labels
and missing labels in this one use case.

Beyond that empty labels should be pruned and not added to storage,
which this also fixes.

Fixes #3841
2018-02-20 11:05:54 +00:00
Julius Volz c3d6abc8e6 Fix some lint errors (#3334)
I left the promql ones and some others untouched as I remember that @fabxc
prefers them that way.
2017-10-23 14:57:30 +01:00
Julius Volz 5da0fd91db Add label matcher tests 2017-10-10 17:13:55 -07:00
Julius Volz 33c1171b9c Don't add anchoring to exported Value matcher field
Instead, just make the anchoring part of the internal regex. This helps because
some users will want to read back the `Value` field and expect it to be the
same as the input value (e.g. some tests in Cortex), or use the value in
another context which is already expected to add its own anchoring, leading to
superfluous double anchoring (such as when we translate matchers into remote
read request matchers).
2017-10-10 10:10:21 -07:00
Brian Brazil 9aa8f822c1 Fix typo 2017-05-24 13:52:45 +01:00
Brian Brazil 5c9a6ce747 Add license to files.
This should fix CI for dev-2.0.
2017-04-19 13:46:22 +01:00
Fabian Reinartz 8ffc851147 Merge branch 'master' into dev-2.0 2017-04-04 15:17:56 +02:00
Fabian Reinartz e94b0899ee rules: fix tests, remove model types 2016-12-29 17:31:14 +01:00
Fabian Reinartz 8b4e4a9d2b notifier: fully use labels.Labels 2016-12-29 16:53:11 +01:00
Fabian Reinartz f8fc1f5bb2 *: migrate ingestion to new batch Appender 2016-12-29 11:03:56 +01:00
Fabian Reinartz 71fe0c58a8 promql: misc fixes 2016-12-28 11:32:15 +01:00
Fabian Reinartz fecf9532b9 *: fix misc compile errors 2016-12-25 11:42:57 +01:00
Fabian Reinartz 0492ddbd4d *: fully decouple tsdb, add new storage interfaces 2016-12-25 01:43:22 +01:00