Commit graph

50 commits

Author SHA1 Message Date
cw9 13cd50c52e Return error on NaN and Inf timestamps for OpenMetrics parser
Signed-off-by: cw9 <chaow929@gmail.com>
2021-06-28 20:55:04 -04:00
Callum Styan 289ba11b79
Add circular in-memory exemplars storage (#6635)
* Add circular in-memory exemplars storage

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>

* Fix some comments, clean up exemplar metrics struct and exemplar tests.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix exemplar query api null vs empty array issue.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>
2021-03-16 15:17:45 +05:30
Julien Pivotto 6c56a1faaa
Testify: move to require (#8122)
* Testify: move to require

Moving testify to require to fail tests early in case of errors.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* More moves

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-29 09:43:23 +00:00
Julien Pivotto 1282d1b39c
Refactor test assertions (#8110)
* Refactor test assertions

This pull request gets rid of assert.True where possible to use
fine-grained assertions.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-27 11:06:53 +01:00
Julien Pivotto 4e5b1722b3
Move away from testutil, refactor imports (#8087)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-22 11:00:08 +02:00
Ben Kochie 269e7c8091
Fix golint issues.
Signed-off-by: Ben Kochie <superq@gmail.com>
2020-03-23 20:38:43 +01:00
Julien Pivotto ed623f69e2
tsdb: don't allow ingesting empty labelsets (#6891)
* tsdb: don't allow ingesting empty labelsets

When we ingest an empty labelset in the head, further blocks can not be
compacted, with the error:

```
level=error ts=2020-02-27T21:26:58.379Z caller=db.go:659 component=tsdb
msg="compaction failed" err="persist head block: write compaction:
add series: out-of-order series added with label set \"{}\" / prev:
\"{}\""
```

We should therefore reject those invalid empty labelsets upfront.

This can be reproduced with the following:

```
cat << END > prometheus.yml
scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 1s
    basic_auth:
      username: test
      password: test
    metric_relabel_configs:
    - regex: ".*"
      action: labeldrop

    static_configs:
    - targets:
      - 127.0.1.1:9090
END
./prometheus --storage.tsdb.min-block-duration=1m
```
And wait a few minutes.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-03-02 07:18:05 +00:00
Julien Pivotto 40b66d29ff
OpenMetrics parse: avoid setting prev token (#6781)
We can avoid setting a prev token in the OM parser. The previous
coundition that checked for prev was unreacheable.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-02-07 07:33:26 +00:00
Julien Pivotto 9398b9ac1b OpenMetrics: force input to end with # EOF (#6505)
Fixes #1169

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-12-24 08:48:28 +00:00
shreyassrivatsan e825282dd1 Add exemplar support to the openmetrics parser (#6292)
* Add exemplar support to the openmetrics parser

Signed-off-by: Shreyas Srivatsan <shreyas@chronosphere.io>
2019-11-19 09:33:30 +00:00
Callum Styan 54e3fc82f1 Add a README.md to textparse package about how to update lexers.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-10-23 23:55:42 -07:00
zhulongcheng 970ef41bb6 Fix exported function comments (#6002)
Signed-off-by: zhulongcheng <zhulongcheng.dev@gmail.com>
2019-09-10 14:45:09 +01:00
Brian Brazil 94b1af1471
Update to disallow new Go 1.13 float formats. (#5984)
Don't inadvertantly expand the acceptable ways to specify
a float.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-09-05 15:18:18 +01:00
Brian Brazil c66bfce8cf
Don't panic on short input. (#5939)
Fixes #5935

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-08-23 15:44:16 +01:00
Bartek Plotka f0863a604e Removed extra tsdb/testutil after merge.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2019-08-14 10:12:32 +01:00
Chris Marchbanks 529ccff07b
Remove all usages of stretchr/testify
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2019-08-08 19:49:27 -06:00
Tariq Ibrahim 8fdfa8abea refine error handling in prometheus (#5388)
i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors.
ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives.
iii) Does away with the use of fmt package for errors in favour of pkg/errors

Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-26 00:01:12 +01:00
Giedrius Statkevičius 10ec2552af promparse: sort all labels when parsing (#5372)
* promparse: sort all labels when parsing

Some label names might start with an uppercase letter in which case it
needs to come before __name__. Without this it means that we are not
maintaining the same invariant that all label names should be sorted in
such cases.

Amend the tests to check this problem automatically. Without the change
the tests do not pass.

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
2019-03-16 10:10:07 +00:00
AixesHunter 9f903fb3f7 github.com/cznic/golex has moved to modernc.org/golex (#5034)
Signed-off-by: aixeshunter <aixeshunter@gmail.com>
2018-12-24 23:59:32 +01:00
Brian Brazil 3fdb92010e Expand OM to OpenMetrics
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-10-18 14:16:41 +01:00
Brian Brazil 9c03e11c2c Hook OpenMetrics parser into scraping.
Extend metadata api to support units.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-10-18 13:58:00 +01:00
Brian Brazil c271ef456a Add an OpenMetrics parser.
This is based on the existing parser, and has slightly better
performance (few %) than it.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-10-18 13:58:00 +01:00
Brian Brazil ffe7efb411 Prepare for multiple text formats
Pass content type down to text parser.

Add layer of indirection in front of text parser,
and rename to avoid future clashes.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-10-18 13:58:00 +01:00
Brian Brazil 70c98a06f1
Handle empty HELP in text parser. (#4444)
Fixes #4427

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-08-01 13:29:58 +01:00
Fabian Reinartz 0ff42e754e pkg/textparse: unescape help string
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-06 05:56:10 -04:00
Fabian Reinartz 5fcc5b6028 pkg/textparse: support null bytes in comments and help
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-04 11:34:34 -04:00
Fabian Reinartz 76a4a46cb0 pkg/textparse: refactor and add metadata handling
Extends the parser to allow retrieving metadata.
The lexer now yields proper tokens that are fed into a hand-written
parser on top.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-05-25 11:27:25 -04:00
Filip Ochnik ec1c6b7c29 textparse: reject empty label names 2018-02-04 17:37:46 +01:00
Julius Volz c3d6abc8e6 Fix some lint errors (#3334)
I left the promql ones and some others untouched as I remember that @fabxc
prefers them that way.
2017-10-23 14:57:30 +01:00
Goutham Veeramachaneni 11ee713ced Handle common escape characters properly. (#2995)
Fixes #2974

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-27 15:15:41 +02:00
Julius Volz c6f41ce391 Allow metric and label names to begin with underscores (#2961)
While this is not recommended, it is allowed in our data model
(https://prometheus.io/docs/concepts/data_model/).

Fixes https://github.com/prometheus/prometheus/issues/2959
2017-07-18 12:35:41 +02:00
Goutham Veeramachaneni b7eddbcd98 textparse: Add fuzzing and fix bug caught
See https://github.com/cznic/golex/issues/11 for info on the bug

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-07 11:12:17 +02:00
Goutham Veeramachaneni 0e78b7e7c8 parser: support spaces everywhere
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-05 15:52:21 +02:00
Brian Brazil dd07f693c8 Single quotes are not valid label value delimiters. 2017-06-20 10:54:39 +01:00
Brian Brazil a6ca391e6e Reject scrapes with invalid utf-8 label values. 2017-06-20 10:54:39 +01:00
Fabian Reinartz bc7aff8cef retrieval: extract scrape cache 2017-05-30 09:37:23 -07:00
Fabian Reinartz a83014f53c retrieval: fix memory leak and consumption for caches 2017-05-26 08:44:24 +02:00
Fabian Reinartz bdc763f95f pkg/textparse: allow null bytes in label values 2017-05-24 14:52:46 +02:00
Fabian Reinartz ea09299ca5 pkg/textparse: handle trailing labels comma (#2752) 2017-05-22 11:15:40 +02:00
Brian Brazil beaa7d5a43 Move consistent NaN logic into the parser. 2017-05-16 18:33:51 +01:00
Fabian Reinartz f7c5d96e84 pkg/textparse: parse metric names with ':' 2017-05-09 12:21:19 +02:00
Fabian Reinartz 377886b371 pkg/textparse: implement timestamp parsing 2017-04-27 17:02:07 +02:00
Brian Brazil 5c9a6ce747 Add license to files.
This should fix CI for dev-2.0.
2017-04-19 13:46:22 +01:00
Fabian Reinartz d9fb57cde4 *: Simplify []byte to string unsafe conversion 2017-03-07 11:41:11 +01:00
Fabian Reinartz d80a3de235 pkg/textparse: add documentation 2017-01-17 08:16:47 +01:00
Fabian Reinartz db48726a6b pkg/textparse: allocate single string per metric 2017-01-16 17:24:00 +01:00
Fabian Reinartz c691895a0f retrieval: cache series references, use pkg/textparse
With this change the scraping caches series references and only
allocates label sets if it has to retrieve a new reference.
pkg/textparse is used to do the conditional parsing and reduce
allocations from 900B/sample to 0 in the general case.
2017-01-16 12:03:57 +01:00
Fabian Reinartz fb3ab9bdb7 pkg/textparse: add more benchmarking, align lex defs 2017-01-15 17:32:57 +01:00
Fabian Reinartz e44d80314d pkg/textparse: add tests and method to retrieve full labels 2017-01-14 19:30:19 +01:00
Fabian Reinartz 091a7f2395 pkg/textparse: add initial text parser 2017-01-14 16:39:04 +01:00