mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-11-15 01:54:06 -08:00
Code Issues Actions 8 Packages Projects Releases Wiki Activity
8996 commits 248 branches 440 tags 657 MiB
Commit graph

91 commits

Author SHA1 Message Date
Levi Harrison 3b5257d869
Changed disabled_features to feature_flags 
Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2021-07-13 22:03:51 -04:00
Filip Petkovski 7c125aa5fb
Promtool: Add support for compaction analysis (#8940) 
* Extend promtool to support compaction analysis

This commit extends the promtool tsdb analyze command to help
troubleshoot high Prometheus disk usage. The command now plots a
distribution of how full chunks are relative to the maximum capacity of
120 samples per chunk.

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>

* Update cmd/promtool/tsdb.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2021-07-02 11:08:52 +01:00
Levi Harrison 90976e7505
Promtool: Add feature flags to unit tests (#8958) 
* Added feature flag support to unit tests

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added/fixed tests

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Addressed review comments

Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2021-06-30 22:43:39 +01:00
Levi Harrison ca1896c15b
Promtool: Validate service discovery files (#8950) 
* Check SD files in promtool

Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2021-06-29 17:32:59 +02:00
Steve Kuznetsov fd6c852567
promtool: backfill: allow configuring block duration (#8919) 
* promtool: backfill: allow configuring block duration

When backfilling large amounts of data across long periods of time, it
may in certain circumstances be useful to use a longer block duration to
increase the efficiency and speed of the backfilling process. This patch
adds a flag --block-duration-power to allow a user to choose the power N
where the block duration is 2^(N+1)h.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* promtool: use sub-tests in backfill testing

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* backfill: add messages to tests for clarity

When someone new breaks a test, seeing "expected: false, got: true" is
really not useful. A nice message helps here.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* backfill: test long block durations

A test that uses a long block duration to write bigger blocks is added.
The check to make sure all blocks are the default duration is removed.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
 2021-06-29 14:53:38 +05:30
Julien Pivotto ba76bceb6b
Merge pull request #8917 from stevekuznetsov/skuznets/silence-backfill 
promtool: backfill: allow silencing output
 2021-06-14 23:27:18 +02:00
Levi Harrison b5f6f8fb36 Switched to go-kit/log 
Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2021-06-11 12:28:36 -04:00
Steve Kuznetsov ee771a2a66
promtool: backfill: allow silencing output 
When using the backfill command to add data to an ephemeral/test
Prometheus instance, it is not important to see which data was added as
it is often generated ahead of time and mostly irrelevant to the
use-case. The current approach prints information about each block that
is written, but does so in a generally inefficient and costly manner.
This patch adds a `--quiet` flag that allows a user to opt out of this
behavior.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
 2021-06-10 15:31:16 -07:00
Jess G 731545ad34
Add documentation for recording rule backfiller (#8674) 
* add docs for rule backfiller

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-04-01 22:38:00 +02:00
Julien Pivotto e635ca834b Add environment variable expansion in external label values 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2021-03-30 01:36:28 +02:00
Björn Rabenstein 9549a15c6f
Merge pull request #7675 from JessicaGreben/jg/11-retroactive-rule-eval 
Add rule importer to backfill
 2021-03-29 19:09:21 +02:00
Ben Kochie f0bccba1c3
Update Go modules for 2.26 (#8636) 
* Update Go modules for 2.26

Bump all Go modules to the latest upstream.

Signed-off-by: Ben Kochie <superq@gmail.com>

* Fix promtool for new client_golang

LabelValues now requires a list of string matchers.

Signed-off-by: Ben Kochie <superq@gmail.com>
 2021-03-24 09:41:12 +00:00
jessicagreben 8de4da3716 add changes per comments, fix tests 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-03-20 12:38:30 -07:00
jessicagreben e3a8132bb3 fix block alignment, add sample alignment 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-03-15 12:44:58 -07:00
jessicagreben 7c26642460 add block alignment and write in 2 hr blocks 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-03-14 10:10:55 -07:00
Pau Freixes b1ac4a45e6 Add num scrapes as tsdb write benchmark command flag 
By default same value that was hardcoded is used, but with the
new flag added the number of scrapes can be increased to any value.

Signed-off-by: Pau Freixes <pfreixes@gmail.com>
 2021-03-10 11:17:07 +01:00
Arthur Silva Sens 537c0aff49
Prometheus and Promtool binaries now print help and usage to stdout (#8542) 
Signed-off-by: ArthurSens <arthursens2005@gmail.com>
 2021-02-25 19:52:34 +01:00
jessicagreben 78e84aed89 resolve merge conflict 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-02-24 09:47:29 -08:00
Łukasz Hryniuk ab41de68b4 Print details of API errors 
Signed-off-by: Łukasz Hryniuk <code@hryniuk.pl>
 2021-02-15 23:42:06 +01:00
Julien Pivotto c1f8bd9944
Merge pull request #8432 from roidelapluie/backfillpanic 
backfill: move checkErr before we close the mmaped file
 2021-02-03 16:32:35 +01:00
Julien Pivotto 9334269f2b backfill: move checkErr before we close the mmaped file 
When printing the error, we still need access to the mmapped byte array
of the file. Therefore, we make sure that we run it before closing the
file.

I could have done something more complex like a defer, or not closing
the file, knowing that we would exit the program anyway. However, I
think that in case we extend this in the future, or this is copy/paster
elsewhere, we should continue closing the file. As it is small enough, I
went for the solution to call the function 3 times instead of playing
with a defer.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2021-02-01 21:18:42 +01:00
Jeremy Albinet 4a1f2c097e Typo on plural in checkRules/checkDuplicates 
Signed-off-by: Jeremy Albinet <jalbinet@synthesio.com>
 2021-02-01 15:43:05 +01:00
Julien Pivotto ac2626757c Update exporter-toolkit to 0.5.0 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2021-01-13 21:49:54 +01:00
Julien Pivotto 003d6451fc Promtool: add web config validation 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-12-29 16:55:29 +01:00
beorn7 651b57b9ab Merge branch 'backfillhr' of git://github.com/roidelapluie/prometheus into review 2020-12-28 16:18:00 +01:00
Julien Pivotto 53480c168d Backfill: print created blocks only, add human-readable option 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-12-23 20:42:30 +01:00
AdaephonBen dca6954b0a
promtool: Add URL scheme when not provided (#7956) 
Signed-off-by: AdaephonBen <ma18btech11011@iith.ac.in>
 2020-12-23 19:52:04 +01:00
jessicagreben cec3515fa3 fix linter 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-30 08:17:51 -08:00
jessicagreben 2e9946e4d7 add test 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-28 07:58:33 -08:00
jessicagreben ac06d0a657 merge master/resolve conflict 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-26 08:43:07 -08:00
jessicagreben ee85c22adb flush samples to disk every 5k samples 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-26 08:30:06 -08:00
Atibhi Agrawal b317b6ab9c
Backfill from OpenMetrics format (#8084) 
* get parser working

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* import file created

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Find min and max ts

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make two passes over file and write to tsdb

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* print error messages

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Max and Min initializer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Start with unit tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reset file read

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* align blocks to two hour range

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add cleanup test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove .ds_store

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add license to import_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Circle CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Refactor code
Move backfill from tsdb to promtool directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix gitignore

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove panic
Rename ContenType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* adjust mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return statement

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go modules

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added unit test for backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix file handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close DB

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Error Handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* inline err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix command line flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add spaces before func
fix pointers

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add defer'd calls

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* move openmetrics.go content to backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changed args to flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add tests for wrong OM files

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added additional tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add comment to warn of func reuse

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Make input required in main.go

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer blockwriter close

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove contentType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove defer from backfilltest

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix defer remove in backfill_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changes to fix CI errors

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go.mod

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change package name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* assert->require

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix format

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix createblock

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* check err for anon func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* update comment

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix for the Flush Bug

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix formatting, comments, names

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* cleanup

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor test to take care of multiple samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* I dont know what I fixed

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests, add test description, print blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* commit after 5000 samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reviews part 1

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Series Count

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove extra func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make timestamp into sec

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Reviews 2

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add Todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fixes

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fixes reviews

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* =0

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove backfill.om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add global err var, remove stuff

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change var name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* sampleLimit pass as parameter

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add test when number of samples greater than batch size

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Change name of batchsize

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* revert export

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add comment, remove newline,consistent err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Modify comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* db.Querier

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add sanity check , get maxt and mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* ci error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* comment change

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* NoError

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>
 2020-11-26 10:37:06 +05:30
jessicagreben 5dd3577424 change name of promtool subcommand to create-blocks-from 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-22 15:05:02 -08:00
jessicagreben 75654715d3 fix panics 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-01 07:54:04 -08:00
jessicagreben 6980bcf671 unexport backfiller 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-10-31 06:40:56 -07:00
jessicagreben 3ed6457dd4 use blockwriter, rm multiwriter code 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-10-31 06:32:07 -07:00
Julien Pivotto 4e5b1722b3
Move away from testutil, refactor imports (#8087) 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-10-22 11:00:08 +02:00
jessicagreben 36ac0b68f1 merge master, fix conflicts 2020-10-17 08:20:21 -07:00
jessicagreben 2e526cf2a7 add output dir parameter 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-09-13 08:38:32 -07:00
jessicagreben dfa510086b add alignment, mv rule importer to promtool dir, add queryRange 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-09-13 08:07:59 -07:00
Julien Pivotto 442b3364d7
Promtool: add evaluation time to instant query (#7829) 
* Promtool: add evaluation time to instant query

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Apply suggestion

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-08-25 11:32:25 +01:00
Andy Bursavich 4e6a94a27d
Invert service discovery dependencies (#7701) 
This also fixes a bug in query_log_file, which now is relative to the config file like all other paths.

Signed-off-by: Andy Bursavich <abursavich@gmail.com>
 2020-08-20 13:48:26 +01:00
jessicagreben 7504b5ce7c add rule importer with tsdb block writer 
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
 2020-07-27 07:44:49 -07:00
Ben Ye 50c261502e
add tsdb cmds into promtool (#6088) 
Signed-off-by: yeya24 <yb532204897@gmail.com>

update tsdb cli in makefile and promu

Signed-off-by: yeya24 <yb532204897@gmail.com>

remove building tsdb bin

Signed-off-by: yeya24 <yb532204897@gmail.com>

remove useless func

Signed-off-by: yeya24 <yb532204897@gmail.com>

refactor analyzeBlock

Signed-off-by: yeya24 <yb532204897@gmail.com>

Fix Makefile

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
 2020-07-23 19:35:50 +01:00
Ben Ye e6ea798c32
promtool range query should exit when fail to parse time (#7505) 
Signed-off-by: yeya24 <yb532204897@gmail.com>
 2020-07-16 23:53:04 +01:00
yeya24 797e48c1a3 support time range in promtool query labels 
Updated prometheus/client_golang and json-iterator/go

Signed-off-by: yeya24 <yb532204897@gmail.com>
 2020-07-03 11:29:39 -04:00
Hongcai Ren c7e82274c6
replace github.com/prometheus/prometheus/testutil/promlint by github.com/prometheus/client_golang/prometheus/testutil/promlint from our codebase (#7209) 
Signed-off-by: RainbowMango <renhongcai@huawei.com>
 2020-05-07 11:34:39 +01:00
Brian Brazil 7646cbca32
Use .UTC everywhere we use time.Unix (#7066) 
time.Unix attaches the local timezone, which can then
leak out (e.g. in the alert json). While this is harmless,
we should be consistent.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2020-03-29 17:35:39 +01:00
Harkishen Singh 84e6459c4d Adds support for line-column numbers for invalid rules, promtool (#6533) 
Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>
 2020-01-15 18:07:54 +00:00
Chris Marchbanks 1d1f64b4bc
Fix Promtool showing false duplicate rule warnings (#6270) 
Alert rules do not use the Record field, so any alerts with the same
labels and different names would be counted as being duplicates.
Promtool will now consider either field when finding duplicates.

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
 2019-11-05 11:22:31 -07:00