Commit graph

127 commits

Author SHA1 Message Date
Brian Brazil 9c03e11c2c Hook OpenMetrics parser into scraping.
Extend metadata api to support units.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-10-18 13:58:00 +01:00
Brian Brazil c271ef456a Add an OpenMetrics parser.
This is based on the existing parser, and has slightly better
performance (few %) than it.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-10-18 13:58:00 +01:00
Brian Brazil ffe7efb411 Prepare for multiple text formats
Pass content type down to text parser.

Add layer of indirection in front of text parser,
and rename to avoid future clashes.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-10-18 13:58:00 +01:00
Tom Wilkie 4c52400708
Limit concurrent remote reads. (#4656)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-09-25 20:07:34 +01:00
Ganesh Vernekar 05726c5ea2 Test template expansion while loading groups (#4537)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2018-09-13 13:55:58 +01:00
Krasi Georgiev 12fe204ea6
move runtime debug funcs in own package (#4494)
To make local debuging with `go run` easyer moved all files into a
dedicate package `runtime`.
This allows running prometheus just by using `go run main.go` instead of
passing mani files like `go run main.go limits_default.go ...`

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-08-22 13:41:11 +03:00
Brian Brazil 70c98a06f1
Handle empty HELP in text parser. (#4444)
Fixes #4427

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2018-08-01 13:29:58 +01:00
Alin Sinpalean 96fb0b2155 Optimize PromQL aggregations (#4248)
* Compute hash of label subsets without creating a LabelSet first.

Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>
2018-07-18 04:56:27 +01:00
Fabian Reinartz 0ff42e754e pkg/textparse: unescape help string
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-06 05:56:10 -04:00
Fabian Reinartz 5fcc5b6028 pkg/textparse: support null bytes in comments and help
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-04 11:34:34 -04:00
Fabian Reinartz 76a4a46cb0 pkg/textparse: refactor and add metadata handling
Extends the parser to allow retrieving metadata.
The lexer now yields proper tokens that are fed into a hand-written
parser on top.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-05-25 11:27:25 -04:00
Elif T. Kuş 57dcdfb15f Rewrote tests with testutil for several test files (#4086)
* promql: Rewrote tests with testutil for functions_test

Signed-off-by: Elif T. Kuş <elifkus@gmail.com>

* pkg/relabel: Rewrote tests with testutil for relabel_test

Signed-off-by: Elif T. Kuş <elifkus@gmail.com>

* discovery/consul: Rewrote tests with testutil for consul_test

Signed-off-by: Elif T. Kuş <elifkus@gmail.com>

* scrape: Rewrote tests with testutil for manager_test

Signed-off-by: Elif T. Kuş <elifkus@gmail.com>
2018-04-27 13:11:16 +01:00
Manos Fokas 25f929b772 Yaml UnmarshalStrict implementation. (#4033)
* Updated yaml vendor package.

* remove checkOverflow duplicate in rulefmt

* remove duplicated HTTPClientConfig.Validate()

* Added yaml static check.
2018-04-04 09:07:39 +01:00
Fabian Reinartz ef567ceb7d
Merge pull request #3835 from krasi-georgiev/pool-package-generalize
Pool package generalize
2018-02-28 14:30:46 +01:00
Krasi Georgiev 933ab8a34e stupid return mistake fix, and dropped the additional assertion check! 2018-02-20 13:32:23 +02:00
Krasi Georgiev 675ce533c9 refactored TestScrapeLoopAppend and added a test for empty labels 2018-02-20 11:05:54 +00:00
Brian Brazil 4addee2bee Fix honor_labels for empty labels, prune empty labels.
The semantics of honor_labels are that if a target exposes
and empty label it will override the target labels. This PR
fixes that by once again distinguishing between empty labels
and missing labels in this one use case.

Beyond that empty labels should be pruned and not added to storage,
which this also fixes.

Fixes #3841
2018-02-20 11:05:54 +00:00
Krasi Georgiev 9b7f0cfa88 rename initialize to make 2018-02-19 14:38:40 +02:00
Krasi Georgiev 9878d66484 exit early and panic for non slice on buffer.Put 2018-02-19 14:31:04 +02:00
Krasi Georgiev f2c5913416 generalize the pool package and refactor scraper to use it. 2018-02-13 22:44:51 +02:00
Filip Ochnik ec1c6b7c29 textparse: reject empty label names 2018-02-04 17:37:46 +01:00
Krasi Georgiev b75428ec19 rename package retrieve to scrape
no fucnctinal changes just renaming retrieval to scrape
2018-02-01 09:55:07 +00:00
Julius Volz 192e665317 rulefmt: Allow parsing from bytes instead of file
This makes the rulefmt package more reusable for users whose rulesets are not
stored in files on disk. Concrete example: Cortex.
2018-01-22 11:46:55 +01:00
Conor Broderick 59d4c539d1 Added rule name to rule error for easier debugging (#3549) 2017-12-06 15:39:06 +00:00
Julius Volz c3d6abc8e6 Fix some lint errors (#3334)
I left the promql ones and some others untouched as I remember that @fabxc
prefers them that way.
2017-10-23 14:57:30 +01:00
Krasi Georgiev 3697075b8f validates that the recording name to include only allowed characters. (#3305) 2017-10-17 10:22:59 +01:00
Julius Volz 5da0fd91db Add label matcher tests 2017-10-10 17:13:55 -07:00
Julius Volz 33c1171b9c Don't add anchoring to exported Value matcher field
Instead, just make the anchoring part of the internal regex. This helps because
some users will want to read back the `Value` field and expect it to be the
same as the input value (e.g. some tests in Cortex), or use the value in
another context which is already expected to add its own anchoring, leading to
superfluous double anchoring (such as when we translate matchers into remote
read request matchers).
2017-10-10 10:10:21 -07:00
Fabian Reinartz d6f6ad67b3 pkg/pool: create bucketed memory pool package. 2017-09-07 14:34:05 +02:00
Goutham Veeramachaneni 11ee713ced Handle common escape characters properly. (#2995)
Fixes #2974

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-27 15:15:41 +02:00
Julius Volz c6f41ce391 Allow metric and label names to begin with underscores (#2961)
While this is not recommended, it is allowed in our data model
(https://prometheus.io/docs/concepts/data_model/).

Fixes https://github.com/prometheus/prometheus/issues/2959
2017-07-18 12:35:41 +02:00
Goutham Veeramachaneni b7eddbcd98 textparse: Add fuzzing and fix bug caught
See https://github.com/cznic/golex/issues/11 for info on the bug

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-07 11:12:17 +02:00
Goutham Veeramachaneni 0e78b7e7c8 parser: support spaces everywhere
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-05 15:52:21 +02:00
Brian Brazil dd07f693c8 Single quotes are not valid label value delimiters. 2017-06-20 10:54:39 +01:00
Brian Brazil a6ca391e6e Reject scrapes with invalid utf-8 label values. 2017-06-20 10:54:39 +01:00
Goutham Veeramachaneni 592cb00c2f
Remove version from RuleGroups
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-19 16:38:46 +05:30
Goutham Veeramachaneni 6b70a4d850
Incorporate PR feedback
* Move fingerprint to Hash()
* Move away from tsdb.MultiError
* 0777 -> 0666 for files
* checkOverflow of extra fields

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-16 16:44:33 +05:30
Goutham Veeramachaneni dc69645e92
Move back to go-yaml
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-16 10:46:21 +05:30
Goutham Veeramachaneni e893c89333
Validate labels and annotations
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-14 15:07:54 +05:30
Goutham Veeramachaneni c884881f7d
Add License Header
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-14 13:34:13 +05:30
Goutham Veeramachaneni 2d1e92513b
Add tests
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-14 13:21:32 +05:30
Goutham Veeramachaneni a48a018368
Make sure groups are unique in a single file
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-14 12:19:21 +05:30
Goutham Veeramachaneni cea1e99f78
Add update-rules command to promtool
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-14 11:38:54 +05:30
Goutham Veeramachaneni e8f55669ea Move rules to new format
Signed-off-by: Goutham Veeramachaneni <goutham@boomerangcommerce.com>
2017-06-12 18:14:39 +05:30
Fabian Reinartz c843a0cd29 pkg/rulefmt: Add rule group parsing 2017-06-07 17:06:23 +02:00
Fabian Reinartz bc7aff8cef retrieval: extract scrape cache 2017-05-30 09:37:23 -07:00
Fabian Reinartz a83014f53c retrieval: fix memory leak and consumption for caches 2017-05-26 08:44:24 +02:00
Fabian Reinartz 10d8b6b633 Merge pull request #2764 from prometheus/nullparse
pkg/textparse: allow null bytes in label values
2017-05-24 15:24:38 +02:00
Fabian Reinartz bdc763f95f pkg/textparse: allow null bytes in label values 2017-05-24 14:52:46 +02:00
Brian Brazil 9aa8f822c1 Fix typo 2017-05-24 13:52:45 +01:00
Fabian Reinartz ea09299ca5 pkg/textparse: handle trailing labels comma (#2752) 2017-05-22 11:15:40 +02:00
Brian Brazil 73049ba79d Make the choice of NaN values clearer.
Also switch stale nan value to one more suitable for expansion.
2017-05-16 18:33:51 +01:00
Brian Brazil a5cf25743c Move stalness check into a function 2017-05-16 18:33:51 +01:00
Brian Brazil beaa7d5a43 Move consistent NaN logic into the parser. 2017-05-16 18:33:51 +01:00
Brian Brazil 76acf7b9b1 Ensure all the NaNs we ingest have the same bit pattern. 2017-05-16 18:33:51 +01:00
Fabian Reinartz f7c5d96e84 pkg/textparse: parse metric names with ':' 2017-05-09 12:21:19 +02:00
Fabian Reinartz 377886b371 pkg/textparse: implement timestamp parsing 2017-04-27 17:02:07 +02:00
Brian Brazil 5c9a6ce747 Add license to files.
This should fix CI for dev-2.0.
2017-04-19 13:46:22 +01:00
Fabian Reinartz 8ffc851147 Merge branch 'master' into dev-2.0 2017-04-04 15:17:56 +02:00
Fabian Reinartz d9fb57cde4 *: Simplify []byte to string unsafe conversion 2017-03-07 11:41:11 +01:00
Fabian Reinartz d80a3de235 pkg/textparse: add documentation 2017-01-17 08:16:47 +01:00
Fabian Reinartz db48726a6b pkg/textparse: allocate single string per metric 2017-01-16 17:24:00 +01:00
Fabian Reinartz c691895a0f retrieval: cache series references, use pkg/textparse
With this change the scraping caches series references and only
allocates label sets if it has to retrieve a new reference.
pkg/textparse is used to do the conditional parsing and reduce
allocations from 900B/sample to 0 in the general case.
2017-01-16 12:03:57 +01:00
Fabian Reinartz fb3ab9bdb7 pkg/textparse: add more benchmarking, align lex defs 2017-01-15 17:32:57 +01:00
Fabian Reinartz e44d80314d pkg/textparse: add tests and method to retrieve full labels 2017-01-14 19:30:19 +01:00
Fabian Reinartz 091a7f2395 pkg/textparse: add initial text parser 2017-01-14 16:39:04 +01:00
Fabian Reinartz e94b0899ee rules: fix tests, remove model types 2016-12-29 17:31:14 +01:00
Fabian Reinartz 8b4e4a9d2b notifier: fully use labels.Labels 2016-12-29 16:53:11 +01:00
Fabian Reinartz f8fc1f5bb2 *: migrate ingestion to new batch Appender 2016-12-29 11:03:56 +01:00
Fabian Reinartz 86cb0f30fd pkg/relabel: add relabel pkg using new labels 2016-12-28 19:04:59 +01:00
Fabian Reinartz 0987a72ec9 pkg/timestamp: create timestamp package 2016-12-28 11:33:00 +01:00
Fabian Reinartz 71fe0c58a8 promql: misc fixes 2016-12-28 11:32:15 +01:00
Fabian Reinartz fecf9532b9 *: fix misc compile errors 2016-12-25 11:42:57 +01:00
Fabian Reinartz 0492ddbd4d *: fully decouple tsdb, add new storage interfaces 2016-12-25 01:43:22 +01:00
Fabian Reinartz 5817cb5bde *: migrate from model.* to promql.* types 2016-12-25 00:37:46 +01:00
Fabian Reinartz 9ea10d5265 promql: use labels.Builder to modify labels 2016-12-24 14:35:24 +01:00
Fabian Reinartz c6cd998905 promql: use local labels, add conversion 2016-12-24 14:01:37 +01:00