mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-25 13:44:05 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
7577 commits 243 branches 444 tags 649 MiB
Commit graph

490 commits

Author SHA1 Message Date
Tobias Guggenmos 9ebf6bd1e6 Remove superfluous blank lines 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-21 13:36:57 +01:00
Tobias Guggenmos 7143d64fc1
Julien's suggestion 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com>
 2020-02-21 13:34:15 +01:00
Tobias Guggenmos 4124828c00 Add test to check that promql.FunctionCalls and parser.Functions contain the same functions 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-21 12:43:30 +01:00
Tobias Guggenmos 4835bbf376
Merge branch 'master' into split_parser 2020-02-19 15:18:13 +01:00
Bartlomiej Plotka 849faa407b Minor fixes. 
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-02-17 18:03:57 +00:00
Bartlomiej Plotka 2cf637fbf5 Addressed comments. 
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-02-17 18:03:57 +00:00
Bartlomiej Plotka 34426766d8 Unify Iterator interfaces. All point to storage now. 
This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things.
All todos I added will be fixed in follow up PRs.

* querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged
with storage interface.go. All imports that.
* querier.SeriesIterator replaced by chunkenc.Iterator
* Added chunkenc.Iterator.Seek method and tests for xor implementation (?)
* Since we properly handle SelectParams for Select methods I adjusted min max
based on that. This should help in terms of performance for queries with functions like offset.
* added Seek to deletedIterator and test.
* storage/tsdb was removed as it was only a unnecessary glue with incompatible structs.

No logic was changed, only different source of abstractions, so no need for benchmarks.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2020-02-17 18:03:54 +00:00
Tobias Guggenmos 2d77611f73 Unbump year number 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:09:23 +01:00
Tobias Guggenmos bf8542a99e Add missing imports to fuzz tests 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:09:23 +01:00
Tobias Guggenmos ec3df53f78 Add nolint comment 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:09:23 +01:00
Tobias Guggenmos b93845d6bf Remove unused function 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:09:23 +01:00
Tobias Guggenmos 43d206f0a1 Add missing file 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:09:23 +01:00
Tobias Guggenmos 6c00f2ffcb Comment fixes 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:09:23 +01:00
Tobias Guggenmos 273ea9093a Fix promql tests 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:09:23 +01:00
Tobias Guggenmos 1360f9ff12 Fix all build errors in promql package 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:09:23 +01:00
Tobias Guggenmos 2164e366ea Fix more identifiers 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:07:53 +01:00
Tobias Guggenmos 5caf7ed6db Fix more identifiers 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:07:53 +01:00
Tobias Guggenmos 9a1366775e Store function implementations independently of their signatures 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:07:53 +01:00
Tobias Guggenmos ff0ea1c1ac Fix more identifiers 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:07:53 +01:00
Tobias Guggenmos 6b1b323558 Export sequenceValue 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:07:53 +01:00
Tobias Guggenmos 228967a507 Fix usages of more things that have moved the package 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:07:27 +01:00
Tobias Guggenmos 4a4817a444 Fix usages of parser.Statement 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:05:21 +01:00
Tobias Guggenmos 2f1113479f Fix usages of ValueType 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:05:21 +01:00
Tobias Guggenmos 65f5b9827d Rename package 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:05:21 +01:00
Tobias Guggenmos 98a9b7c90c Remove engine specific types 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:05:21 +01:00
Tobias Guggenmos fab2373752 Add everything the parser needs to build 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:05:21 +01:00
Tobias Guggenmos e7c2716f55 Move files 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-02-17 16:05:21 +01:00
Zhou Hao e628fd7735
fix comments spelling (#6829) 
Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>
 2020-02-17 12:45:11 +01:00
Björn Rabenstein af04cb22c8
Merge pull request #6821 from prometheus/release-2.16 
Release 2.16
 2020-02-14 13:10:14 +01:00
Julien Pivotto ff0003e072
Make lookbackDelta a option of QueryEngine (#6746) 
* Make lookbackDelta a option of QueryEngine

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* julius' suggestion

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* remove trivial getter

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Assume lookback delta is always > 0

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* add debug log

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* don't expose loopback delta

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Specify that lookack delta is also used in federation

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Fix federation test

While we have added some logic to the promql engine to keep it backwards
compatible and have a 5 minute loopback by default, the web/ package is
likely to really be internal to Prometheus and we should not add the
same kind of heuritstics here.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* loopback delta: Fix debug log

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-02-10 00:58:23 +01:00
Julien Pivotto cbd0eec9fc Avoid /1000 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-02-08 16:30:09 +01:00
Julien Pivotto 881dde505a promql: fix promql query log step unit 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-02-08 16:26:56 +01:00
Julien Pivotto 9adad8ad30 Remove MaxConcurrent from the PromQL engine opts (#6712) 
Since we use ActiveQueryTracker to check for concurrency in
d992c36b3a it does not make sense to keep
the MaxConcurrent value as an option of the PromQL engine.

This pull request removes it from the PromQL engine options, sets the
max concurrent metric to -1 if there is no active query tracker, and use
the value of the active query tracker otherwise.

It removes dead code and also will inform people who import the promql
package that we made that change, as it breaks the EngineOpts struct.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-01-28 20:38:49 +00:00
Brian Brazil 61262159c4 Simplify benchmark given the new API 
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2020-01-28 14:38:09 +00:00
Brian Brazil 38d32e0686 Don't sort postings if we only have one block. 
Sorting the heads postings can be quite slow.
We only need sorted series when merging with another
querier, so only sort then.
This will make big queries that only touch the head faster,
though queries that touch both the head and a block will still
be the same speed. This probably won't help much with graphing
unless the range is under an hour, however it should make most
recording rules faster.

Add gaurantee that remote read streaming produces sorted series.

PromQL benchmarks for histograms show only 2-3% improvement, but
they're only over 1k series.

benchmark                                                old ns/op      new ns/op      delta
BenchmarkQuerierSelect/Head/1of1000000-4                 1375486282     507657736      -63.09%
BenchmarkQuerierSelect/Head/10of1000000-4                1387859004     507769850      -63.41%
BenchmarkQuerierSelect/Head/100of1000000-4               1387087935     506029110      -63.52%
BenchmarkQuerierSelect/Head/1000of1000000-4              1386869064     504521986      -63.62%
BenchmarkQuerierSelect/Head/10000of1000000-4             1386213685     505210422      -63.55%
BenchmarkQuerierSelect/Head/100000of1000000-4            1392754988     529842406      -61.96%
BenchmarkQuerierSelect/Head/1000000of1000000-4           1569414722     725059506      -53.80%
BenchmarkQuerierSelect/SortedHead/1of1000000-4           1381019902     1370495863     -0.76%
BenchmarkQuerierSelect/SortedHead/10of1000000-4          1375696209     1366789468     -0.65%
BenchmarkQuerierSelect/SortedHead/100of1000000-4         1386009422     1364519297     -1.55%
BenchmarkQuerierSelect/SortedHead/1000of1000000-4        1377700532     1364486191     -0.96%
BenchmarkQuerierSelect/SortedHead/10000of1000000-4       1383539536     1369545314     -1.01%
BenchmarkQuerierSelect/SortedHead/100000of1000000-4      1410089163     1394731339     -1.09%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-4     1634744148     1581554956     -3.25%
BenchmarkQuerierSelect/Block/1of1000000-4                881741242      879839470      -0.22%
BenchmarkQuerierSelect/Block/10of1000000-4               880381562      882846038      +0.28%
BenchmarkQuerierSelect/Block/100of1000000-4              887519357      881016916      -0.73%
BenchmarkQuerierSelect/Block/1000of1000000-4             902194205      883433524      -2.08%
BenchmarkQuerierSelect/Block/10000of1000000-4            892321964      885130170      -0.81%
BenchmarkQuerierSelect/Block/100000of1000000-4           938604466      933527150      -0.54%
BenchmarkQuerierSelect/Block/1000000of1000000-4          1313510845     1295881124     -1.34%

benchmark                                                old allocs     new allocs     delta
BenchmarkQuerierSelect/Head/1of1000000-4                 4000056        4000018        -0.00%
BenchmarkQuerierSelect/Head/10of1000000-4                4000074        4000036        -0.00%
BenchmarkQuerierSelect/Head/100of1000000-4               4000254        4000216        -0.00%
BenchmarkQuerierSelect/Head/1000of1000000-4              4002054        4002016        -0.00%
BenchmarkQuerierSelect/Head/10000of1000000-4             4020054        4020016        -0.00%
BenchmarkQuerierSelect/Head/100000of1000000-4            4200054        4200016        -0.00%
BenchmarkQuerierSelect/Head/1000000of1000000-4           6000054        6000016        -0.00%
BenchmarkQuerierSelect/SortedHead/1of1000000-4           4000071        4000071        +0.00%
BenchmarkQuerierSelect/SortedHead/10of1000000-4          4000089        4000089        +0.00%
BenchmarkQuerierSelect/SortedHead/100of1000000-4         4000269        4000269        +0.00%
BenchmarkQuerierSelect/SortedHead/1000of1000000-4        4002069        4002069        +0.00%
BenchmarkQuerierSelect/SortedHead/10000of1000000-4       4020069        4020069        +0.00%
BenchmarkQuerierSelect/SortedHead/100000of1000000-4      4200069        4200069        +0.00%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-4     6000069        6000069        +0.00%
BenchmarkQuerierSelect/Block/1of1000000-4                6000023        6000022        -0.00%
BenchmarkQuerierSelect/Block/10of1000000-4               6000059        6000058        -0.00%
BenchmarkQuerierSelect/Block/100of1000000-4              6000419        6000418        -0.00%
BenchmarkQuerierSelect/Block/1000of1000000-4             6004019        6004018        -0.00%
BenchmarkQuerierSelect/Block/10000of1000000-4            6040019        6040018        -0.00%
BenchmarkQuerierSelect/Block/100000of1000000-4           6400019        6400018        -0.00%
BenchmarkQuerierSelect/Block/1000000of1000000-4          10000020       10000019       -0.00%

benchmark                                                old bytes     new bytes     delta
BenchmarkQuerierSelect/Head/1of1000000-4                 229192200     176001176     -23.21%
BenchmarkQuerierSelect/Head/10of1000000-4                229193352     176002328     -23.21%
BenchmarkQuerierSelect/Head/100of1000000-4               229204872     176013848     -23.21%
BenchmarkQuerierSelect/Head/1000of1000000-4              229320072     176129048     -23.20%
BenchmarkQuerierSelect/Head/10000of1000000-4             230472072     177281048     -23.08%
BenchmarkQuerierSelect/Head/100000of1000000-4            241992072     188801048     -21.98%
BenchmarkQuerierSelect/Head/1000000of1000000-4           357192072     304001048     -14.89%
BenchmarkQuerierSelect/SortedHead/1of1000000-4           229193928     229193928     +0.00%
BenchmarkQuerierSelect/SortedHead/10of1000000-4          229195080     229195080     +0.00%
BenchmarkQuerierSelect/SortedHead/100of1000000-4         229206600     229206600     +0.00%
BenchmarkQuerierSelect/SortedHead/1000of1000000-4        229321800     229321800     +0.00%
BenchmarkQuerierSelect/SortedHead/10000of1000000-4       230473800     230473800     +0.00%
BenchmarkQuerierSelect/SortedHead/100000of1000000-4      241993800     241993800     +0.00%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-4     357193800     357193800     +0.00%
BenchmarkQuerierSelect/Block/1of1000000-4                227201516     227201500     -0.00%
BenchmarkQuerierSelect/Block/10of1000000-4               227202924     227202908     -0.00%
BenchmarkQuerierSelect/Block/100of1000000-4              227217036     227217020     -0.00%
BenchmarkQuerierSelect/Block/1000of1000000-4             227358156     227358140     -0.00%
BenchmarkQuerierSelect/Block/10000of1000000-4            228769356     228769340     -0.00%
BenchmarkQuerierSelect/Block/100000of1000000-4           242881356     242881340     -0.00%
BenchmarkQuerierSelect/Block/1000000of1000000-4          384001616     384001600     -0.00%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2020-01-28 09:14:56 +00:00
Julien Pivotto d992c36b3a promql: make active query tracker context-aware (#6701) 
* promql: make query logger context-aware
* Remove gate
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-01-27 22:29:44 +00:00
Julien Pivotto cf42888e4d Fix order of testutil.Equals (#6695) 
Equals takes the expected value as first parameter, and the actual value
as second parameter.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-01-27 12:21:59 +00:00
Julien Pivotto 5f27ac3583 Refactor query log fields (#6694) 
* Refactor query log fields

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-01-27 09:53:10 +00:00
Julien Pivotto 3fbeee07fa Fix flaky windows test (#6677) 
The windows clock is sometime off by 25ms, and as precise as 15ms.

Let's give it more time to avoid flaky tests.

Fix #6672

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-01-22 10:09:31 +00:00
Julien Pivotto 2b2eb79e8b Add windows tests for query logger (#6653) 
* Add windows tests
* Do not rely on time.Time in timer

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-01-20 13:17:11 +00:00
Tobias Guggenmos 2aacd807b3 PromQL: Various small improvements in the parser (#6652) 
* Move check for empty VectorSelector to typeChecking
* Move check for twice set metric name to typeChecking
* Make child of MatrixSelector a general Node
* rename checkType to checkAST
* Rename fail to addParseErr
* Remove trailing whitespace

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-01-17 15:16:58 +00:00
Tobias Guggenmos f5eed7ae0a PromQL: Fix panic in parser (#6650) 
Fixes #6649.

The crash is fixed here, was caused because some  AST sanity checks were performed on the syntax tree while assembling it. In case of previous parsing errors this could lead to undefined behaviour.

The fix is to move the checks to the typechecking phase, which runs only when a syntax tree was assembled without there being parsing errors.

There are other places, where similiar checks are performed while assembling the syntax tree. It might be a good idea to move those to the typechecking phase, too. Should I do this in the same or a separate PR?

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-01-17 13:06:26 +00:00
Tobias Guggenmos 2ca25d1c96 PromQL: Don't panic on syntax errors (#6643) 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-01-16 15:20:20 +00:00
Tobias Guggenmos d642570924 PromQL: Use PositionRange in parser errors (#6634) 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-01-15 20:01:49 +00:00
Tobias Guggenmos 3a204be6b7 PromQL: Fix string and parentheses handling in engine (#6612) 
* WIP: PromQL: Allow engine to return strings

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Add test suggested by @roidelapluie

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Fix typo in React UI

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Fix parenthesis handling for functions and aggregator params

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Add more tests

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>

* Fix React UI test

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-01-15 18:31:58 +01:00
Tobias Guggenmos 0c8e9ef09e PromQL: Add position metadata to the AST (#6615) 
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
 2020-01-14 16:12:15 +00:00
Tobias Guggenmos 64194f7d45 PromQL: AST: Make VectorSelector Children of MatrixSelector (#6590) 
Make Vector selectors children of Matrix Selectors

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-01-10 14:25:41 +00:00
Julien Pivotto 3885562587 Query Logging styling (#6594) 
- Fix Json vs JSON in activequerylogger
- Fix SetQueryLogger always returns nil

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-01-09 21:11:39 +00:00
Tobias Guggenmos 6534ce843f PromQL: Use a sync.Pool for the generated parser structure (#6591) 
* PromQL: Use a sync.Pool for the generated parser structure

The generated PromQL parser allocates a struct about 4kb in size on every run.

This puts a high load on the garbage collector.

To reduce that load, a sync.Pool is used to recycle these structures.

On small queries this makes parsing 2-3 times faster.

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
 2020-01-09 17:36:13 +00:00
Guangwen Feng 835c27a76b Fix typo in comment for func Lex (#6588) 
Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
 2020-01-09 11:27:26 +00:00