Commit graph

514 commits

Author SHA1 Message Date
B++ d6374ae1b6
Return NaN for histogram_quantile when buckets have 0 observations (#7318)
Signed-off-by: jberny <f.bernardi89@gmail.com>
2020-06-01 09:40:39 +01:00
Julien Pivotto 58c445e6ef
Fuzz: limit input size (#7317)
We know that fuzzParseExpr and fuzzParseMetricSelector make use of heavy
things like regexes, which take a fairly big amount of memory.

OSS-Fuzz does not offer a proper way to increase the memory [1], therefore
we limit the input size [2].

[1] https://google.github.io/oss-fuzz/faq/#how-do-you-handle-timeouts-and-ooms
[2] https://google.github.io/oss-fuzz/getting-started/new-project-guide/#input-size

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-05-31 09:42:56 +02:00
Brian Brazil 3932a7149f
Correctly track points no longer used by matrixIterSlice's slice. (#7307)
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-05-28 13:36:30 +01:00
Callum Styan 5bb7f00d00
change labelset comparison in promql engine to avoid false positive during detection of duplicates (#7058)
* Use go1.14 new hash/maphash to hash both RHS and LHS instead of XOR'ing
which has been resulting in hash collisions.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Refactor engine labelset signature generation, just use labels.Labels
instead of hashes.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address review comments; function comments + store result of
lhs.String+rhs.String as key.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Replace all signatureFunc usage with signatureFuncString.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Make optimizations to labels String function and generation of rhs+lhs
as string in resultMetric.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Use separate string functions that don't use strconv just for engine
maps.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Use a byte invalid separator instead of quoting and have a buffer
attached to EvalNodeHelper instead of using a global pool in the labels
package.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address more review comments, labels has a function that now builds a
byte slice without turning it into a string.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Use two different non-ascii hex codes as byte separators between labels
and between sets of labels when building bytes of a Labels struct.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* We only need the 2nd byte invalid sep. at the beginning of a
labels.Bytes

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2020-05-12 14:03:15 -07:00
Ben Ye 1e4e37144d
Fixed wrongly handled not ready TSDB on web and API. (#7182)
* fix federate endpoint panic

Signed-off-by: yeya24 <yb532204897@gmail.com>

* Fixed all cases of not ready TSDB being wrongly handled.

* Fixed issue for federation.
* Ensured this will never happen again thanks to interfaces
* Fixes same issue for stats.
* Added tests for readiness.
* Fixed bug in stats. It was:
   status.MaxTime = db.Head().MaxTime()
   status.MinTime = db.Head().MaxTime()


Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed Brian's comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed Brian's comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-04-29 17:16:14 +01:00
Vasily Sliouniaev 0393b188c9
Add Jaeger (#7148)
* Trace remote read

Signed-off-by: vas <vasily.sliouniaev@jet.com>

* Use jaeger

Signed-off-by: vas <vasily.sliouniaev@jet.com>
2020-04-23 02:05:55 +02:00
Julien Pivotto 1f6f8e60ee promql/parser: Cleanup generatedParserResult accross reuse
Reusing the same generatedParserResult ends up in strange panics:
See #7131 and #7127.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-04-16 01:51:08 +02:00
Marek Slabicki 8224ddec23
Capitalizing first letter of all log lines (#7043)
Signed-off-by: Marek Slabicki <thaniri@gmail.com>
2020-04-11 09:22:18 +01:00
Brian Brazil 7646cbca32
Use .UTC everywhere we use time.Unix (#7066)
time.Unix attaches the local timezone, which can then
leak out (e.g. in the alert json). While this is harmless,
we should be consistent.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-03-29 17:35:39 +01:00
Ben Kochie 269e7c8091
Fix golint issues.
Signed-off-by: Ben Kochie <superq@gmail.com>
2020-03-23 20:38:43 +01:00
johncming bdc45c2b9e
remove unused code. (#7019)
Signed-off-by: johncming <johncming@yahoo.com>
2020-03-21 07:15:51 +00:00
Björn Rabenstein 1da83305be
Merge pull request #7009 from prometheus/release-2.17
Merge release-2.17 into master
2020-03-19 13:46:28 +01:00
Tobias Guggenmos 012161d90d
PromQL: Fix lexer error handling (#6958)
* PromQL: Fix lexer error handling

This fixes bugs in the handling of lexer errors that are only noticeable for users of the language server and caused https://github.com/prometheus-community/promql-langserver/issues/104 .

Signed-off-by: Tobias Guggenmos <tobias.guggenmos@uni-ulm.de>

* Add test for error position ranges

Signed-off-by: Tobias Guggenmos <tobias.guggenmos@uni-ulm.de>
2020-03-16 15:47:47 +01:00
Björn Rabenstein a28fa010ee
TSDB: Extract parts out of populateSeries (#6983)
This addresses fabxc's TODO.

More importantly, it now properly defers the
querier.Close(). Previously, if a panic happened after creation of the
querier within the populateSeries function, querier.Close() was never called.

The latter was responsible for #6977.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-03-14 09:03:40 +01:00
Bartlomiej Plotka fe802f29c9 storage: Removed SelectSorted method; Simplified interface; Added requirement for remote read to sort response.
This is technically BREAKING CHANGE, but it was like this from the beginning: I just notice that we rely in
Prometheus on remote read being sorted. This is because we use selected data from remote reads in MergeSeriesSet
which rely on sorting.

I found during work on https://github.com/prometheus/prometheus/pull/5882 that
we do so many repetitions because of this, for not good reason. I think
I found a good balance between convenience and readability with just one method.
Smaller the interface = better.

Also I don't know what TestSelectSorted was testing, but now it's testing sorting.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-03-13 13:06:25 +00:00
Björn Rabenstein bc703b6456
Use struct{} as underlying type for context keys (#6965)
This is an alternative to #6963.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-03-11 15:05:35 +01:00
Julien Pivotto 5ddd1dcf0f
Fix panic when parsing varags (#6940)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-03-08 13:09:24 +01:00
Tobias Guggenmos 1dbd799354
PromQL: Fix regression tests (#6935)
This PR fixes the regression tests for the issue fixed in #6931 .

The reason for that is that all of the invalid queries that triggered the regression have become more or less valid syntax in #6933 (they might still fail typechecking).

Signed-off-by: Tobias Guggenmos <tobias.guggenmos@uni-ulm.de>
2020-03-06 08:17:01 +00:00
Brian Brazil 44ad28dd5e
PromQL: Allow more keywords as metric names (#6933)
* Allow more keywords as metric names
* Add documentation about forbidden keywords

Signed-off-by: Tobias Guggenmos <tobias.guggenmos@uni-ulm.de>
2020-03-05 13:20:53 +00:00
Brian Brazil 7164b58945
PromQL: Fix parser panic (#6931)
Signed-off-by: Tobias Guggenmos <tobias.guggenmos@uni-ulm.de>
2020-03-05 08:03:38 +00:00
李国忠 2bf4952049
remove Unused parameter 'sf' in calcTrendValue function (#6900)
Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>
2020-03-03 08:23:27 +00:00
LongKB 82f7ed208b
Remove some duplicated words (#6882)
Signed-off-by: Pham Duc Hanh <hanhpd@fujitsu.com>
2020-02-27 07:08:31 +01:00
Tobias Guggenmos 3d74fcfa6a Bartek's suggestions
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-25 13:57:30 +01:00
Tobias Guggenmos f9db320e5a Look up function call in all cases
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-24 13:45:03 +01:00
Tobias Guggenmos 9ebf6bd1e6 Remove superfluous blank lines
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-21 13:36:57 +01:00
Tobias Guggenmos 7143d64fc1
Julien's suggestion
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com>
2020-02-21 13:34:15 +01:00
Tobias Guggenmos 4124828c00 Add test to check that promql.FunctionCalls and parser.Functions contain the same functions
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-21 12:43:30 +01:00
Tobias Guggenmos 4835bbf376
Merge branch 'master' into split_parser 2020-02-19 15:18:13 +01:00
Bartlomiej Plotka 849faa407b Minor fixes.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:57 +00:00
Bartlomiej Plotka 2cf637fbf5 Addressed comments.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:57 +00:00
Bartlomiej Plotka 34426766d8 Unify Iterator interfaces. All point to storage now.
This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things.
All todos I added will be fixed in follow up PRs.

* querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged
with storage interface.go. All imports that.
* querier.SeriesIterator replaced by chunkenc.Iterator
* Added chunkenc.Iterator.Seek method and tests for xor implementation (?)
* Since we properly handle SelectParams for Select methods I adjusted min max
based on that. This should help in terms of performance for queries with functions like offset.
* added Seek to deletedIterator and test.
* storage/tsdb was removed as it was only a unnecessary glue with incompatible structs.

No logic was changed, only different source of abstractions, so no need for benchmarks.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:54 +00:00
Tobias Guggenmos 2d77611f73 Unbump year number
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos bf8542a99e Add missing imports to fuzz tests
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos ec3df53f78 Add nolint comment
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos b93845d6bf Remove unused function
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos 43d206f0a1 Add missing file
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos 6c00f2ffcb Comment fixes
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos 273ea9093a Fix promql tests
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos 1360f9ff12 Fix all build errors in promql package
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos 2164e366ea Fix more identifiers
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos 5caf7ed6db Fix more identifiers
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos 9a1366775e Store function implementations independently of their signatures
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos ff0ea1c1ac Fix more identifiers
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos 6b1b323558 Export sequenceValue
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos 228967a507 Fix usages of more things that have moved the package
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:27 +01:00
Tobias Guggenmos 4a4817a444 Fix usages of parser.Statement
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Tobias Guggenmos 2f1113479f Fix usages of ValueType
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Tobias Guggenmos 65f5b9827d Rename package
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Tobias Guggenmos 98a9b7c90c Remove engine specific types
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Tobias Guggenmos fab2373752 Add everything the parser needs to build
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00