Commit graph

485 commits

Author SHA1 Message Date
Ganesh Vernekar 04ad56d9b8
Append sparse histograms into the Head block (#9013)
* Append sparse histograms into the Head block

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Add AtHistogram() to Iterator interface. Make HistoChunk conform to Chunk interface.

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-29 20:08:46 +05:30
Ganesh Vernekar 64bea6999e
HistogramAppender interface for sparse histograms (#9007)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-28 20:30:55 +05:30
Ben Kochie 7cb55d5732
Merge pull request #8802 from mwasilew2/yaml-linting
Adds yamllinting to Makefile.common
2021-06-24 15:59:35 +02:00
Julien Pivotto ba76bceb6b
Merge pull request #8917 from stevekuznetsov/skuznets/silence-backfill
promtool: backfill: allow silencing output
2021-06-14 23:27:18 +02:00
Michal Wasilewski 3f686cad8b
fixes yamllint errors
Signed-off-by: Michal Wasilewski <mwasilewski@gmx.com>
2021-06-12 12:47:47 +02:00
Levi Harrison b5f6f8fb36 Switched to go-kit/log
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-11 12:28:36 -04:00
Steve Kuznetsov ee771a2a66
promtool: backfill: allow silencing output
When using the backfill command to add data to an ephemeral/test
Prometheus instance, it is not important to see which data was added as
it is often generated ahead of time and mostly irrelevant to the
use-case. The current approach prints information about each block that
is written, but does so in a generally inefficient and costly manner.
This patch adds a `--quiet` flag that allows a user to opt out of this
behavior.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
2021-06-10 15:31:16 -07:00
Levi Harrison 7bc11dcb06
React UI: Add Starting Screen (#8662)
* Added walreplay API endpoint

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added starting page to react-ui

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Documented the new endpoint

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed typos

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Co-authored-by: Julius Volz <julius.volz@gmail.com>

* Removed logo

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed isResponding to isUnexpected

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed width of progress bar

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed width of progress bar

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added DB stats object

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Updated starting page to work with new fields

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 2)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 3)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (and also implementing a method this time) (pt. 4)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (and also implementing a method this time) (pt. 5)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed const to let

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 6)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Remove SetStats method

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added comma

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed api

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed to triple equals

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed data response types

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Don't return pointer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed version

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed interface issue

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed pointer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed copying lock value error

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Co-authored-by: Julius Volz <julius.volz@gmail.com>
2021-06-05 15:29:32 +01:00
Levi Harrison 17ea8d006a
Added external URL access
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-05-30 23:35:26 -04:00
Bartlomiej Plotka 80545bfb2e
Instrumented circular exemplar storage. (#8712)
* Instrumented circular storage.

Fixes: https://github.com/prometheus/prometheus/issues/8708
Fixes: https://github.com/prometheus/prometheus/issues/8707

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed CB.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed Julien comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed Callum comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2021-04-16 13:44:53 +01:00
nberkley f9e2dd0697
Add support for smaller block chunk segment allocations (#8478)
* Add support for --storage.tsdb.max-chunk-size to suport small chunks for space limited prometheus instances.

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update tsdb/compact.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update tsdb/db.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update cmd/prometheus/main.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Change naming scheme to

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Add a lower bound to --storage.tsdb.max-block-chunk-segment-size

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update storage.md to explain what a chunk segment is

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Apply suggestions from code review

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Force tests

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Fix code style

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-04-15 14:25:01 +05:30
Julien Pivotto ae73a6296a
Merge pull request #8683 from cuirunxing-hub/main
typos correct
2021-04-02 20:14:55 +02:00
cuirunxing-hub 57bc2e94e2 typos correct
Signed-off-by: cuirunxing-hub <cuirunxing@inspur.com>
2021-04-02 09:03:00 +08:00
Jess G 731545ad34
Add documentation for recording rule backfiller (#8674)
* add docs for rule backfiller

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-04-01 22:38:00 +02:00
Julien Pivotto e635ca834b Add environment variable expansion in external label values
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-30 01:36:28 +02:00
Björn Rabenstein 9549a15c6f
Merge pull request #7675 from JessicaGreben/jg/11-retroactive-rule-eval
Add rule importer to backfill
2021-03-29 19:09:21 +02:00
jessicagreben 896c828bb5 close writer after flush
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-03-29 06:45:12 -07:00
jessicagreben d89a1d999f add log with start/end times, close blocks before end of func
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-03-28 12:13:58 -07:00
Ben Kochie f0bccba1c3
Update Go modules for 2.26 (#8636)
* Update Go modules for 2.26

Bump all Go modules to the latest upstream.

Signed-off-by: Ben Kochie <superq@gmail.com>

* Fix promtool for new client_golang

LabelValues now requires a list of string matchers.

Signed-off-by: Ben Kochie <superq@gmail.com>
2021-03-24 09:41:12 +00:00
Julien Pivotto c0c36b1155
Improve promql-negative-offset docs (#8631)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-22 10:16:43 +01:00
jessicagreben 8de4da3716 add changes per comments, fix tests
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-03-20 12:38:30 -07:00
Callum Styan 289ba11b79
Add circular in-memory exemplars storage (#6635)
* Add circular in-memory exemplars storage

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>

* Fix some comments, clean up exemplar metrics struct and exemplar tests.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix exemplar query api null vs empty array issue.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>
2021-03-16 15:17:45 +05:30
jessicagreben e3a8132bb3 fix block alignment, add sample alignment
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-03-15 12:44:58 -07:00
jessicagreben 7c26642460 add block alignment and write in 2 hr blocks
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-03-14 10:10:55 -07:00
Julien Pivotto 63ea88af82
Merge pull request #8575 from pfreixes/add-scrapes-parameter
Add num scrapes as tsdb write benchmark command flag
2021-03-11 13:09:50 +01:00
Pau Freixes b1ac4a45e6 Add num scrapes as tsdb write benchmark command flag
By default same value that was hardcoded is used, but with the
new flag added the number of scrapes can be increased to any value.

Signed-off-by: Pau Freixes <pfreixes@gmail.com>
2021-03-10 11:17:07 +01:00
Julien Pivotto ad5ed416ba
Merge pull request #8487 from pschou/dev_neg_offset
allow negative offset
2021-03-08 22:18:45 +01:00
Julien Pivotto 5742a18590 Fix subqueries with default resolution in promql unit tests
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-07 09:20:04 +01:00
jessicagreben 9fc53b7edf fix appender.Add -> appender.Append
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-03-01 05:49:49 -08:00
Arthur Silva Sens 537c0aff49
Prometheus and Promtool binaries now print help and usage to stdout (#8542)
Signed-off-by: ArthurSens <arthursens2005@gmail.com>
2021-02-25 19:52:34 +01:00
jessicagreben 78e84aed89 resolve merge conflict
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-02-24 09:47:29 -08:00
jessicagreben f2db9dc722 add multi rule integration tests
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-02-24 09:42:31 -08:00
pschou f80b52be69
Merge branch 'main' into dev_neg_offset 2021-02-23 20:52:57 -05:00
schou 22cd48868a adding feature flag, promql-negative-offset
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:25:56 -05:00
Julien Pivotto 8c8de46003
Merge pull request #8036 from dgl/promtool-alert-err
promtool: Don't end alert tests early, in some failure situations
2021-02-20 22:35:00 +01:00
Tom Wilkie 7369561305
Combine Appender.Add and AddFast into a single Append method. (#8489)
This moves the label lookup into TSDB, whilst still keeping the cached-ref optimisation for repeated Appends.

This makes the API easier to consume and implement.  In particular this change is motivated by the scrape-time-aggregation work, which I don't think is possible to implement without it as it needs access to label values.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2021-02-18 17:37:00 +05:30
Julien Pivotto 1fac1c783b
Merge pull request #8504 from rbauduin/require_alertname
promtool: alert_rule_test items require alertname
2021-02-17 22:07:52 +01:00
Julien Pivotto 2d172d0896
Merge pull request #8508 from prometheus/release-2.25
Merge back release 2.25
2021-02-17 16:26:34 +01:00
Raphael Bauduin a7d64cad21 promtool: alert_rule_test items require alertname
Accepting alert_rule_test without alertname is confusing as it will
always pass with empty exp_alerts, and never with non-empty exp_alerts.

Signed-off-by: Raphael Bauduin <raphael.bauduin@tessares.net>
2021-02-17 16:23:12 +01:00
Ganesh Vernekar c4536fa28c
Increase block writer size for backfilling
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2021-02-17 15:45:41 +05:30
Julien Pivotto a419b75abd
Merge pull request #8485 from hryniuk/promtool-query-errors-details
Print details of API errors received by promtool
2021-02-16 22:47:08 +01:00
Łukasz Hryniuk ab41de68b4 Print details of API errors
Signed-off-by: Łukasz Hryniuk <code@hryniuk.pl>
2021-02-15 23:42:06 +01:00
David Leadbeater 3e30f72af1 promtool: Add more negative alert tests
Signed-off-by: David Leadbeater <dgl@dgl.cx>
2021-02-15 17:00:49 +00:00
Julien Pivotto e29b47b39e
Merge pull request #8440 from mishamo/master
Add optional name property to testgroup for better test failure output
2021-02-09 21:23:24 +01:00
misha 1c3e7b4241 Use strings.Builder for neater error formatting
Signed-off-by: misha <DL-OTTCloudPlatform-Nova@bskyb.internal>
2021-02-09 15:00:26 +00:00
Tom Wilkie d479151f1f Various enhancements and refactorings for remote write receiver:
- Remove unrelated changes
- Refactor code out of the API module - that is already getting pretty crowded.
- Don't track reference for AddFast in remote write.  This has the potential to consume unlimited server-side memory if a malicious client pushes a different label set for every series.  For now, its easier and safer to always use the 'slow' path.
- Return 400 on out of order samples.
- Use remote.DecodeWriteRequest in the remote write adapters.
- Put this behing the 'remote-write-server' feature flag
- Add some (very) basic docs.
- Used named return & add test for commit error propagation

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2021-02-08 20:41:23 +00:00
fuling 72475b8a0c [ENHANCEMENT] remote storage:Add default api implementation of remote write
Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>
2021-02-07 18:12:48 +00:00
misha c2c5aeb16b Add optional name property to testgroup for better test failure output
Signed-off-by: misha <DL-OTTCloudPlatform-Nova@bskyb.internal>
2021-02-04 10:07:22 +00:00
Julien Pivotto c1f8bd9944
Merge pull request #8432 from roidelapluie/backfillpanic
backfill: move checkErr before we close the mmaped file
2021-02-03 16:32:35 +01:00
Julien Pivotto 9334269f2b backfill: move checkErr before we close the mmaped file
When printing the error, we still need access to the mmapped byte array
of the file. Therefore, we make sure that we run it before closing the
file.

I could have done something more complex like a defer, or not closing
the file, knowing that we would exit the program anyway. However, I
think that in case we extend this in the future, or this is copy/paster
elsewhere, we should continue closing the file. As it is small enough, I
went for the solution to call the function 3 times instead of playing
with a defer.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-02-01 21:18:42 +01:00