Commit graph

8714 commits

Author SHA1 Message Date
Nicolas Lamirault aa1ca13025
Add: Custom tags and prefix in Prometheus Mixin (#8287)
* Add: custom tags and prefix

Signed-off-by: Nicolas Lamirault <nicolas.lamirault@gmail.com>

* Fix: fmt

Signed-off-by: Nicolas Lamirault <nicolas.lamirault@gmail.com>
2020-12-16 18:49:06 +01:00
fcddk b270a1d6bf import strutil package
import strutil package

Signed-off-by: root <likerj@inspur.com>
2020-12-16 16:27:03 +08:00
fcddk 617c56f55a add endponits labels metadata
add endponits labels metadata

Signed-off-by: root <likerj@inspur.com>
2020-12-16 16:07:49 +08:00
Alexey Shumkin 73ddf603af
discovery/kubernetes: Fix valid label selector causing config error
Label selector can be
"set-based"(https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#set-based-requirement)
but such a selector causes Prometheus start failure with the "unexpected
error: parsing YAML file ...: invalid selector: 'foo in (bar,baz)';
can't understand 'baz)'"-like error.

This is caused by the `fields.ParseSelector(string)` function that
simply splits an expression as a CSV-list, so a comma confuses such a
parsing method and lead to the error.

Use `labels.Parse(string)` to use a valid lexer to parse a selector
expression.

Closes #8284.

Signed-off-by: Alexey Shumkin <Alex.Crezoff@gmail.com>
2020-12-16 10:56:01 +03:00
dependabot[bot] abe85ce052
Bump ini from 1.3.5 to 1.3.8 in /web/ui/react-app
Bumps [ini](https://github.com/isaacs/ini) from 1.3.5 to 1.3.8.
- [Release notes](https://github.com/isaacs/ini/releases)
- [Commits](https://github.com/isaacs/ini/compare/v1.3.5...v1.3.8)

Signed-off-by: dependabot[bot] <support@github.com>
2020-12-15 22:50:00 +00:00
Julien Pivotto 8dc53c2218
Api: Verify that at every matcher in api/v1/series is not empty (#8288)
Fixed #8286

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-15 17:24:57 +00:00
Atibhi Agrawal 0dd6e0cb76
change string(model.metricnamelabel) to model.metricnamelabel (#8289)
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-15 20:17:31 +05:30
Atibhi Agrawal 7a519fb294
add fullstop (#8290)
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-15 20:13:44 +05:30
Atibhi Agrawal c08009139a
Add docs for backfill (#8230)
* Add docs for backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-10 16:29:44 +01:00
lzhfromustc 27a6e1e174
test: add buffer to channel to avoid goroutine leak (#8274)
Signed-off-by: lzhfromustc <lzhfromustc@gmail.com>
2020-12-10 09:09:21 +00:00
Julien Pivotto 66f47e116e
Merge pull request #8269 from roidelapluie/fuzz
Remove fuzzit, add oss-fuzz on PR
2020-12-09 21:58:04 +01:00
Julien Pivotto 7bc48123e5 Restrict fuzzing on go paths
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 21:37:14 +01:00
Julien Pivotto 4a4ea309b6 Add oss-fuzz on PR's
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 21:13:18 +01:00
Julien Pivotto c666879e3f Remove fuzzit
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 21:10:31 +01:00
Björn Rabenstein e13f4a5f6e
Merge pull request #8232 from roidelapluie/eof
TSDB backfill: enforce # EOF
2020-12-09 13:22:03 +01:00
Julien Pivotto 7957731339 Inline defer
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 09:23:39 +01:00
Julien Pivotto 82b5f1d8b1 Backfill: Use mmap to reuse parser code
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-08 23:48:31 +01:00
Björn Rabenstein 511511324a
Merge pull request #8235 from Allex1/master
Update remote-write grafana mixin
2020-12-08 14:50:47 +01:00
Julien Pivotto 87e142c02f
remote_write documentation: fix blank lines (#8264)
The blank line is not efficiently placed in the remote_write config,
affecting readability.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-08 14:27:50 +01:00
Björn Rabenstein 404b2980cf
Merge pull request #8254 from prometheus/beorn7/mixin
mixin: Add a capability to exclude non-prod AM instances
2020-12-08 13:33:27 +01:00
Arthur Silva Sens 64a106c5dd
Logging added for when compaction takes more than the block time range (#8151)
* Logging added for when compaction takes more than the block time range

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Log only if no errors were already logged

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Log duration as human readable string

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Move logging from compactHead() to Compact()

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Compute duration of all head compactions plus wal truncation

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Remove named return added os first commits

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Address nits

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Change miliseconds to seconds to make fuzzit tests happy

Signed-off-by: ArthurSens <arthursens2005@gmail.com>
2020-12-07 21:29:43 +00:00
beorn7 553f904f2d mixin: Add a capability to exclude non-prod AM instances
Signed-off-by: beorn7 <beorn@grafana.com>
2020-12-03 20:59:53 +01:00
Thibault Jamet f055690b34
Clarify: all rules in a group are concomitant (#8248)
Improve the documentation to clarify the differences beetween rules in a
group and outside a group.

Signed-off-by: Thibault Jamet <tjamet@users.noreply.github.com>
2020-12-03 10:32:10 +00:00
Chris Aniszczyk c21e0b713d
Add CodeQL Security Scanning
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
2020-12-02 15:48:39 -08:00
Mitsuo Heijo 8b64b70fe4
Guard closing quitCh with sync.Once to prevent double close (#8242)
* Guard closing quitCh with sync.Once to prevent double close

Signed-off-by: Mitsuo Heijo <mitsuo.heijo@gmail.com>
2020-12-02 09:39:54 +01:00
birca 3ec4161575 Update remote-write grafana mixin
Signed-off-by: birca <birca@adobe.com>
2020-12-02 09:50:15 +02:00
Matt Berther acee998df6
CNAME responses can occur with "Type: A" dns_sd_config requests (#8216)
Signed-off-by: Matt Berther <mattberther@users.noreply.github.com>
2020-12-01 09:32:15 +00:00
jessicagreben e32e4fcc53 fix unit test
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2020-11-30 11:02:45 -08:00
jessicagreben cec3515fa3 fix linter
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2020-11-30 08:17:51 -08:00
johncming a6e18916ab
tsdb: Remove duplicate variables. (#8239)
Signed-off-by: johncming <johncming@yahoo.com>
2020-11-30 08:55:33 +00:00
Mike DeSimone 759150878c
fix trivial typos (#8238)
Signed-off-by: Mike DeSimone <michael.2.desimone@gmail.com>
2020-11-29 01:23:10 +00:00
jessicagreben 2e9946e4d7 add test
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2020-11-28 07:58:33 -08:00
Julien Pivotto 868184b33a
Merge pull request #8234 from prometheus/release-2.23
Merge release 2.23 to master
2020-11-28 00:41:46 +01:00
jessicagreben ac06d0a657 merge master/resolve conflict
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2020-11-26 08:43:07 -08:00
jessicagreben ee85c22adb flush samples to disk every 5k samples
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2020-11-26 08:30:06 -08:00
Ganesh Vernekar 26d89b4b07
Cut v2.23.0 (#8223)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-26 15:41:54 +05:30
Atibhi Agrawal b317b6ab9c
Backfill from OpenMetrics format (#8084)
* get parser working

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* import file created

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Find min and max ts

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make two passes over file and write to tsdb

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* print error messages

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Max and Min initializer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Start with unit tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reset file read

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* align blocks to two hour range

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add cleanup test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove .ds_store

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add license to import_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Circle CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Refactor code
Move backfill from tsdb to promtool directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix gitignore

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove panic
Rename ContenType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* adjust mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return statement

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go modules

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added unit test for backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix file handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close DB

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Error Handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* inline err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix command line flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add spaces before func
fix pointers

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add defer'd calls

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* move openmetrics.go content to backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changed args to flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add tests for wrong OM files

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added additional tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add comment to warn of func reuse

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Make input required in main.go

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer blockwriter close

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove contentType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove defer from backfilltest

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix defer remove in backfill_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changes to fix CI errors

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go.mod

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change package name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* assert->require

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix format

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix createblock

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* check err for anon func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* update comment

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix for the Flush Bug

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix formatting, comments, names

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* cleanup

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor test to take care of multiple samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* I dont know what I fixed

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests, add test description, print blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* commit after 5000 samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reviews part 1

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Series Count

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove extra func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make timestamp into sec

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Reviews 2

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add Todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fixes

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fixes reviews

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* =0

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove backfill.om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add global err var, remove stuff

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change var name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* sampleLimit pass as parameter

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add test when number of samples greater than batch size

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Change name of batchsize

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* revert export

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add comment, remove newline,consistent err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Modify comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* db.Querier

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add sanity check , get maxt and mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* ci error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* comment change

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* NoError

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-11-26 10:37:06 +05:30
Ganesh Vernekar dff967286e
Set the min time of Head properly after truncation (#8212)
* Set the min time of Head properly after truncation

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix lint

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Enhance compaction plan logic for completely deleted small block

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-25 18:33:30 +05:30
Julien Pivotto f50dde7031
Remove Krasi as TSDB maintainer (#8213)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-11-24 20:24:25 +01:00
Ganesh Vernekar 9972e69c43
Cut v2.23.0-rc.0 (#8206)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-23 14:52:37 +05:30
jessicagreben 5dd3577424 change name of promtool subcommand to create-blocks-from
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2020-11-22 15:05:02 -08:00
jessicagreben 19dee0a569 add name and labels to metric, eval all rules for each block
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2020-11-22 14:24:38 -08:00
George Brighton 76cd5f4c7f
Fix typo in EC2 SD docs (#8210)
Signed-off-by: George Brighton <george@gebn.co.uk>
2020-11-20 19:22:03 +00:00
Muhammad Falak R Wani 587fd740be
discovery: ec2: expose IPv6 as label (#7086)
* discovery: ec2: expose IPv6 as label

Signed-off-by: Muhammad Falak R Wani <falakreyaz@gmail.com>
2020-11-20 17:13:52 +00:00
Justin Toh 5f103aaecc
docs: Fix "time duration" link (#8209)
Signed-off-by: Justin Toh <tohjustin@hotmail.com>
2020-11-20 12:25:45 +00:00
Chris Marchbanks c8f810083d
Cache label name and last value when adding series (#8192)
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2020-11-19 11:18:12 -07:00
Ganesh Vernekar c2ce236de1
Upgrade react deps (#8207)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-19 17:10:29 +01:00
gotjosh 4eca4dffb8
Allow metric metadata to be propagated via Remote Write. (#6815)
* Introduce a metadata watcher

Similarly to the WAL watcher, its purpose is to observe the scrape manager and pull metadata. Then, send it to a remote storage.

Signed-off-by: gotjosh <josue@grafana.com>

* Additional fixes after rebasing.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Rework samples/metadata metrics.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Use more descriptive variable names in MetadataWatcher collect.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix issues caused during rebasing.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix missing metric add and unneeded config code.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address some review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix metrics and docs

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Replace assert with require

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Bring back max_samples_per_send metric

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

Co-authored-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-19 20:53:03 +05:30
Ganesh Vernekar 3ad25a6dc3
Merge pull request #8204 from prometheus/release-2.22
Backport v2.22.2 to master
2020-11-19 19:57:52 +05:30
Ganesh Vernekar 8cc94fac93
Update dependencies for v2.23 (#8203)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-19 19:32:19 +05:30