Commit graph

8549 commits

Author SHA1 Message Date
Alfred Krohmer 4efca5ab70
EC2 service discovery: reuse EC2 client instead of recreating it every time (#8311)
This also caches credentials that are obtained e.g. via IRSA on AWS EKS.
Previously, every refresh cycle would request the credentials again.

Signed-off-by: Alfred Krohmer <alfred.krohmer@logmein.com>
2020-12-22 15:47:44 +00:00
Ganesh Vernekar faa1554aa1
Don't call runtime.GC() after compaction (#8276)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-12-22 14:44:17 +00:00
Ben Ye caa173d2aa
Support matchers for Labels API (#8301)
Signed-off-by: Ben Ye <yb532204897@gmail.com>
Co-authored-by: Erik Klockare <eklockare@gmail.com>
2020-12-22 11:02:19 +00:00
Julien Pivotto e5478983c4
Merge pull request #8299 from roidelapluie/fixdocker
Fix docker image build
2020-12-16 21:49:53 +01:00
Julien Pivotto 041d0bcfd6 Fix docker image build
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-16 21:21:22 +01:00
Nicolas Lamirault aa1ca13025
Add: Custom tags and prefix in Prometheus Mixin (#8287)
* Add: custom tags and prefix

Signed-off-by: Nicolas Lamirault <nicolas.lamirault@gmail.com>

* Fix: fmt

Signed-off-by: Nicolas Lamirault <nicolas.lamirault@gmail.com>
2020-12-16 18:49:06 +01:00
fcddk b270a1d6bf import strutil package
import strutil package

Signed-off-by: root <likerj@inspur.com>
2020-12-16 16:27:03 +08:00
fcddk 617c56f55a add endponits labels metadata
add endponits labels metadata

Signed-off-by: root <likerj@inspur.com>
2020-12-16 16:07:49 +08:00
Alexey Shumkin 73ddf603af
discovery/kubernetes: Fix valid label selector causing config error
Label selector can be
"set-based"(https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#set-based-requirement)
but such a selector causes Prometheus start failure with the "unexpected
error: parsing YAML file ...: invalid selector: 'foo in (bar,baz)';
can't understand 'baz)'"-like error.

This is caused by the `fields.ParseSelector(string)` function that
simply splits an expression as a CSV-list, so a comma confuses such a
parsing method and lead to the error.

Use `labels.Parse(string)` to use a valid lexer to parse a selector
expression.

Closes #8284.

Signed-off-by: Alexey Shumkin <Alex.Crezoff@gmail.com>
2020-12-16 10:56:01 +03:00
dependabot[bot] abe85ce052
Bump ini from 1.3.5 to 1.3.8 in /web/ui/react-app
Bumps [ini](https://github.com/isaacs/ini) from 1.3.5 to 1.3.8.
- [Release notes](https://github.com/isaacs/ini/releases)
- [Commits](https://github.com/isaacs/ini/compare/v1.3.5...v1.3.8)

Signed-off-by: dependabot[bot] <support@github.com>
2020-12-15 22:50:00 +00:00
Julien Pivotto 8dc53c2218
Api: Verify that at every matcher in api/v1/series is not empty (#8288)
Fixed #8286

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-15 17:24:57 +00:00
Atibhi Agrawal 0dd6e0cb76
change string(model.metricnamelabel) to model.metricnamelabel (#8289)
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-15 20:17:31 +05:30
Atibhi Agrawal 7a519fb294
add fullstop (#8290)
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-15 20:13:44 +05:30
Atibhi Agrawal c08009139a
Add docs for backfill (#8230)
* Add docs for backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-10 16:29:44 +01:00
lzhfromustc 27a6e1e174
test: add buffer to channel to avoid goroutine leak (#8274)
Signed-off-by: lzhfromustc <lzhfromustc@gmail.com>
2020-12-10 09:09:21 +00:00
Julien Pivotto 66f47e116e
Merge pull request #8269 from roidelapluie/fuzz
Remove fuzzit, add oss-fuzz on PR
2020-12-09 21:58:04 +01:00
Julien Pivotto 7bc48123e5 Restrict fuzzing on go paths
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 21:37:14 +01:00
Julien Pivotto 4a4ea309b6 Add oss-fuzz on PR's
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 21:13:18 +01:00
Julien Pivotto c666879e3f Remove fuzzit
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 21:10:31 +01:00
Björn Rabenstein e13f4a5f6e
Merge pull request #8232 from roidelapluie/eof
TSDB backfill: enforce # EOF
2020-12-09 13:22:03 +01:00
Julien Pivotto 7957731339 Inline defer
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 09:23:39 +01:00
Julien Pivotto 82b5f1d8b1 Backfill: Use mmap to reuse parser code
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-08 23:48:31 +01:00
Björn Rabenstein 511511324a
Merge pull request #8235 from Allex1/master
Update remote-write grafana mixin
2020-12-08 14:50:47 +01:00
Julien Pivotto 87e142c02f
remote_write documentation: fix blank lines (#8264)
The blank line is not efficiently placed in the remote_write config,
affecting readability.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-08 14:27:50 +01:00
Björn Rabenstein 404b2980cf
Merge pull request #8254 from prometheus/beorn7/mixin
mixin: Add a capability to exclude non-prod AM instances
2020-12-08 13:33:27 +01:00
Arthur Silva Sens 64a106c5dd
Logging added for when compaction takes more than the block time range (#8151)
* Logging added for when compaction takes more than the block time range

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Log only if no errors were already logged

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Log duration as human readable string

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Move logging from compactHead() to Compact()

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Compute duration of all head compactions plus wal truncation

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Remove named return added os first commits

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Address nits

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Change miliseconds to seconds to make fuzzit tests happy

Signed-off-by: ArthurSens <arthursens2005@gmail.com>
2020-12-07 21:29:43 +00:00
beorn7 553f904f2d mixin: Add a capability to exclude non-prod AM instances
Signed-off-by: beorn7 <beorn@grafana.com>
2020-12-03 20:59:53 +01:00
Thibault Jamet f055690b34
Clarify: all rules in a group are concomitant (#8248)
Improve the documentation to clarify the differences beetween rules in a
group and outside a group.

Signed-off-by: Thibault Jamet <tjamet@users.noreply.github.com>
2020-12-03 10:32:10 +00:00
Chris Aniszczyk c21e0b713d
Add CodeQL Security Scanning
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
2020-12-02 15:48:39 -08:00
Mitsuo Heijo 8b64b70fe4
Guard closing quitCh with sync.Once to prevent double close (#8242)
* Guard closing quitCh with sync.Once to prevent double close

Signed-off-by: Mitsuo Heijo <mitsuo.heijo@gmail.com>
2020-12-02 09:39:54 +01:00
birca 3ec4161575 Update remote-write grafana mixin
Signed-off-by: birca <birca@adobe.com>
2020-12-02 09:50:15 +02:00
Matt Berther acee998df6
CNAME responses can occur with "Type: A" dns_sd_config requests (#8216)
Signed-off-by: Matt Berther <mattberther@users.noreply.github.com>
2020-12-01 09:32:15 +00:00
johncming a6e18916ab
tsdb: Remove duplicate variables. (#8239)
Signed-off-by: johncming <johncming@yahoo.com>
2020-11-30 08:55:33 +00:00
Mike DeSimone 759150878c
fix trivial typos (#8238)
Signed-off-by: Mike DeSimone <michael.2.desimone@gmail.com>
2020-11-29 01:23:10 +00:00
Julien Pivotto 868184b33a
Merge pull request #8234 from prometheus/release-2.23
Merge release 2.23 to master
2020-11-28 00:41:46 +01:00
Ganesh Vernekar 26d89b4b07
Cut v2.23.0 (#8223)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-26 15:41:54 +05:30
Atibhi Agrawal b317b6ab9c
Backfill from OpenMetrics format (#8084)
* get parser working

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* import file created

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Find min and max ts

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make two passes over file and write to tsdb

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* print error messages

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Max and Min initializer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Start with unit tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reset file read

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* align blocks to two hour range

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add cleanup test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove .ds_store

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add license to import_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Circle CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Refactor code
Move backfill from tsdb to promtool directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix gitignore

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove panic
Rename ContenType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* adjust mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return statement

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go modules

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added unit test for backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix file handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close DB

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Error Handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* inline err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix command line flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add spaces before func
fix pointers

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add defer'd calls

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* move openmetrics.go content to backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changed args to flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add tests for wrong OM files

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added additional tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add comment to warn of func reuse

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Make input required in main.go

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer blockwriter close

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove contentType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove defer from backfilltest

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix defer remove in backfill_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changes to fix CI errors

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go.mod

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change package name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* assert->require

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix format

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix createblock

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* check err for anon func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* update comment

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix for the Flush Bug

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix formatting, comments, names

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* cleanup

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor test to take care of multiple samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* I dont know what I fixed

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests, add test description, print blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* commit after 5000 samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reviews part 1

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Series Count

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove extra func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make timestamp into sec

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Reviews 2

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add Todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fixes

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fixes reviews

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* =0

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove backfill.om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add global err var, remove stuff

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change var name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* sampleLimit pass as parameter

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add test when number of samples greater than batch size

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Change name of batchsize

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* revert export

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add comment, remove newline,consistent err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Modify comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* db.Querier

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add sanity check , get maxt and mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* ci error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* comment change

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* NoError

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-11-26 10:37:06 +05:30
Ganesh Vernekar dff967286e
Set the min time of Head properly after truncation (#8212)
* Set the min time of Head properly after truncation

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix lint

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Enhance compaction plan logic for completely deleted small block

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-25 18:33:30 +05:30
Julien Pivotto f50dde7031
Remove Krasi as TSDB maintainer (#8213)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-11-24 20:24:25 +01:00
Ganesh Vernekar 9972e69c43
Cut v2.23.0-rc.0 (#8206)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-23 14:52:37 +05:30
George Brighton 76cd5f4c7f
Fix typo in EC2 SD docs (#8210)
Signed-off-by: George Brighton <george@gebn.co.uk>
2020-11-20 19:22:03 +00:00
Muhammad Falak R Wani 587fd740be
discovery: ec2: expose IPv6 as label (#7086)
* discovery: ec2: expose IPv6 as label

Signed-off-by: Muhammad Falak R Wani <falakreyaz@gmail.com>
2020-11-20 17:13:52 +00:00
Justin Toh 5f103aaecc
docs: Fix "time duration" link (#8209)
Signed-off-by: Justin Toh <tohjustin@hotmail.com>
2020-11-20 12:25:45 +00:00
Chris Marchbanks c8f810083d
Cache label name and last value when adding series (#8192)
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2020-11-19 11:18:12 -07:00
Ganesh Vernekar c2ce236de1
Upgrade react deps (#8207)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-19 17:10:29 +01:00
gotjosh 4eca4dffb8
Allow metric metadata to be propagated via Remote Write. (#6815)
* Introduce a metadata watcher

Similarly to the WAL watcher, its purpose is to observe the scrape manager and pull metadata. Then, send it to a remote storage.

Signed-off-by: gotjosh <josue@grafana.com>

* Additional fixes after rebasing.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Rework samples/metadata metrics.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Use more descriptive variable names in MetadataWatcher collect.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix issues caused during rebasing.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix missing metric add and unneeded config code.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address some review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix metrics and docs

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Replace assert with require

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Bring back max_samples_per_send metric

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

Co-authored-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-19 20:53:03 +05:30
Ganesh Vernekar 3ad25a6dc3
Merge pull request #8204 from prometheus/release-2.22
Backport v2.22.2 to master
2020-11-19 19:57:52 +05:30
Ganesh Vernekar 8cc94fac93
Update dependencies for v2.23 (#8203)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-19 19:32:19 +05:30
Marco Pracucci db19e05d93
Add option to customise head chunks write buffer size (#8201)
* Add option to customise head chunks write buffer size

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Fixed tests

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2020-11-19 18:30:47 +05:30
Peter Wu 1797192f02
Fix the alerting rules name description (#7083) (#8197)
commit 9875afc491 changed the type from
metric names to label values, we might as well adjust the description.
The alternative is to revert that commit and restrict names of alerting
rules again even if that was not really enforced.

Signed-off-by: Peter Wu <pwu@cloudflare.com>
2020-11-18 19:29:01 +00:00