Commit graph

734 commits

Author SHA1 Message Date
Arve Knudsen 85df1d4013 Merge remote-tracking branch 'prometheus/main' into chore/sync-prometheus
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-21 09:19:23 +02:00
Bryan Boreham 5ecea3c840 promtool: fix compile error from bad merge
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-09-20 16:34:20 +00:00
Ben Ye c78124427e
Support specifying series matchers to analyze tsdb (#12842)
* support specifying series matchers to analyze tsdb

Signed-off-by: Ben Ye <benye@amazon.com>

* fix cli docs

Signed-off-by: Ben Ye <benye@amazon.com>

---------

Signed-off-by: Ben Ye <benye@amazon.com>
2023-09-20 11:37:32 +01:00
Paschalis Tsilias c173cd57c9
Add a header to count retried remote write requests (#12729)
Header name is `Retry-Attempt`, only set when >0.

Signed-off-by: Marc Tuduri <marctc@protonmail.com>
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
2023-09-20 11:11:03 +01:00
Arve Knudsen e48d4e5835 Merge remote-tracking branch 'prometheus/main' into chore/sync-prometheus
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-18 09:29:42 +02:00
zenador 69edd8709b
Add warnings (and annotations) to PromQL query results (#12152)
Return annotations (warnings and infos) from PromQL queries

This generalizes the warnings we have already used before (but only for problems with remote read) as "annotations".

Annotations can be warnings or infos (the latter could be false positives). We do not treat them different in the API for now and return them all as "warnings". It would be easy to distinguish them and return infos separately, should that appear useful in the future.

The new annotations are then used to create a lot of warnings or infos during PromQL evaluations. Partially these are things we have wanted for a long time (e.g. inform the user that they have applied `rate` to a metric that doesn't look like a counter), but the new native histograms have created even more needs for those annotations (e.g. if a query tries to aggregate float numbers with histograms).

The annotations added here are not yet complete. A prominent example would be a warning about a range too short for a rate calculation. But such a warnings is more tricky to create with good fidelity and we will tackle it later.

Another TODO is to take annotations into account when evaluating recording rules.

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-09-14 18:57:31 +02:00
Arve Knudsen 156222cc50
Add context argument to LabelQuerier.LabelValues (#12665)
Add context argument to LabelQuerier.LabelValues and
LabelQuerier.SortedLabelValues.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-14 16:02:04 +02:00
Arve Knudsen a964349e97
Add context argument to LabelQuerier.LabelNames (#12666)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-14 10:39:51 +02:00
Arve Knudsen 4451ba10b4
Add context argument to IndexReader.Postings (#12667)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-13 17:45:06 +02:00
Arve Knudsen 6ef9ed0bc3
Add context argument to DB.Delete (#12834)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-13 15:43:06 +02:00
Arve Knudsen 6daee89e5f
Add context argument to Querier.Select (#12660)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2023-09-12 12:37:38 +02:00
Dimitar Dimitrov 77ac7ad40a
Merge remote-tracking branch 'upstream/main' into dimitar/pull-upstream 2023-09-05 16:19:00 +02:00
Ziqi Zhao eaaa21aa7f
promtool tsdb dump support native histogram (#12775)
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-09-01 13:21:52 +10:00
Gregor Zeitlinger f01718262a
Unit tests for native histograms (#12668)
promql: Extend testing framework to support native histograms

This includes both the internal testing framework as well as the rules unit test feature of promtool.

This also adds a bunch of basic tests. Many of the code level tests can now be converted to tests within the framework, and more tests can be added easily.

---------

Signed-off-by: Harold Dost <h.dost@criteo.com>
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
Signed-off-by: Stephen Lang <stephen.lang@grafana.com>
Co-authored-by: Harold Dost <h.dost@criteo.com>
Co-authored-by: Stephen Lang <stephen.lang@grafana.com>
Co-authored-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
2023-08-25 23:35:42 +02:00
Sylvain Rabot 4399959f79
Remove native histograms / memory snapshot restriction
Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>
2023-08-18 15:15:55 +02:00
Oleg Zaytsev a54cb4744d Merge branch 'main' into prometheus-2023-07-31-76dd9b547 2023-08-04 09:46:03 +02:00
Goutham Veeramachaneni ad4f514e66
Add OTLP Ingestion endpoint (#12571)
* Add OTLP Ingestion endpoint

We copy files from the otel-collector-contrib. See the README in
`storage/remote/otlptranslator/README.md`.

This supersedes: https://github.com/prometheus/prometheus/pull/11965

Signed-off-by: gouthamve <gouthamve@gmail.com>

* Return a 200 OK

It is what the OTEL Golang SDK expect :(

https://github.com/open-telemetry/opentelemetry-go/issues/4363

Signed-off-by: Goutham <gouthamve@gmail.com>

---------

Signed-off-by: gouthamve <gouthamve@gmail.com>
Signed-off-by: Goutham <gouthamve@gmail.com>
2023-07-28 12:35:28 +02:00
Jeanette Tan 8035c04624 Merge remote-tracking branch 'upstream/main'
Minor conflicts:
rules/manager.go
tsdb/compact.go
tsdb/db.go
go.mod
2023-07-19 21:40:27 +08:00
Julien Pivotto b3b669fd9a Add experimental flag and docs
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-07-12 22:33:49 +02:00
Rob Skillington e1ace8d00e Add PromQL format and label matcher set/delete commands to promtool
Signed-off-by: Rob Skillington <rob@chronosphere.io>
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-07-12 22:33:44 +02:00
Justin Lei 32d87282ad
Add Zstandard compression option for wlog (#11666)
Snappy remains as the default compression but there is now a flag to switch 
the compression algorithm.

Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-07-11 14:57:57 +02:00
Bryan Boreham 578e2b6a3f re-order imports for linter
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-08 12:39:33 +00:00
François Gouteroux e86fe245b6 feat refactoring checkrules func
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-07-04 15:01:01 +00:00
François Gouteroux 7f1f842eba fix: apply suggested changes
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-07-04 15:01:01 +00:00
François Gouteroux 488ece2899 fix: apply suggested changes
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-07-04 15:01:01 +00:00
François Gouteroux ce00e8c7a8 promtool: read from stdin if no filenames are provided in check rules
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-07-04 15:01:01 +00:00
François Gouteroux e0ab445e9a fix: apply suggested changes
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-07-04 15:01:01 +00:00
François Gouteroux 9a55e301f7 feat: enhance stdin check and add tests parsing error
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-07-04 15:01:01 +00:00
François Gouteroux 430a664cf4 feat: make push metrics labels generic and repeatable
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-07-04 15:01:01 +00:00
François Gouteroux 25d352e6e9 feat: add suggested changes, tests, and stdin support
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-07-04 15:01:01 +00:00
François Gouteroux 5b9accd2b5 feat(promtool): add push metrics command
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-07-04 15:01:01 +00:00
tyltr 96717579d2 typo
Signed-off-by: tyltr <tylitianrui@126.com>
2023-07-04 15:01:01 +00:00
Nidhey Nitin Indurkar e970f70ced Feat: Get block by id directly on promtool analyze & get latest block if ID not provided (#12031)
* feat: analyze latest block or block by ID in CLI (promtool)

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>

* address remarks

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

* address latest review comments

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

---------

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>
Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>
2023-07-04 13:39:01 +00:00
Bryan Boreham 5255bf06ad Replace sort.Slice with faster slices.SortFunc
The generic version is more efficient.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-02 22:17:08 +00:00
Julien Pivotto 1214d314c3
Merge pull request #12225 from fgouteroux/feat/promtool_check_rules_stdin
promtool: read from stdin if no filenames are provided in check rules
2023-06-27 13:22:00 +02:00
Julien Pivotto 771f512757
Merge pull request #12299 from fgouteroux/promtool_push_metrics_cmd
feat(promtool): add push metrics command
2023-06-27 10:42:14 +02:00
François Gouteroux 58d38c4c56 fix: apply suggested changes
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-06-27 09:30:39 +02:00
tyltr 0941ea4afc typo
Signed-off-by: tyltr <tylitianrui@126.com>
2023-06-16 11:09:19 +08:00
François Gouteroux f676d4a756 feat refactoring checkrules func
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-06-01 18:04:53 +02:00
Nidhey Nitin Indurkar a8772a4178
Feat: Get block by id directly on promtool analyze & get latest block if ID not provided (#12031)
* feat: analyze latest block or block by ID in CLI (promtool)

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>

* address remarks

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

* address latest review comments

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

---------

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>
Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>
2023-06-01 17:13:09 +05:30
François Gouteroux 6ae4a46845 feat: enhance stdin check and add tests parsing error
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-06-01 10:28:55 +02:00
François Gouteroux 4341b98eb2 fix: apply suggested changes
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-05-28 19:55:00 +02:00
François Gouteroux 934c5ddb8d feat: make push metrics labels generic and repeatable
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-05-24 11:33:07 +02:00
François Gouteroux 3524a16aa0 feat: add suggested changes, tests, and stdin support
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-05-23 11:15:29 +02:00
Jeanette Tan 1be0816b46 Merge remote-tracking branch 'upstream/main' 2023-05-23 00:20:36 +08:00
Baskar Shanmugam 905a0bd63a
Added 'limit' query parameter support to /api/v1/status/tsdb endpoint (#12336)
* Added 'topN' query parameter support to /api/v1/status/tsdb endpoint

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

* Updated query parameter for tsdb status to 'limit'

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

* Corrected Stats() parameter name from topN to limit

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

* Fixed p.Stats CI failure

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

---------

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>
2023-05-22 14:37:07 +02:00
Callum Styan 0d2108ad79
[tsdb] re-implement WAL watcher to read via a "notification" channel (#11949)
* WIP implement WAL watcher reading via notifications over a channel from
the TSDB code

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Notify via head appenders Commit (finished all WAL logging) rather than
on each WAL Log call

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix misspelled Notify plus add a metric for dropped Write notifications

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Update tests to handle new notification pattern

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* this test maybe needs more time on windows?

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* does this test need more time on windows as well?

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* read timeout is already a time.Duration

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* remove mistakenly commited benchmark data files

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* address some review feedback

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* fix missed changes from previous commit

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix issues from wrapper function

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* try fixing race condition in test by allowing tests to overwrite the
read ticker timeout instead of calling the Notify function

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* fix linting

Signed-off-by: Callum Styan <callumstyan@gmail.com>

---------

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2023-05-15 12:31:49 -07:00
Björn Rabenstein 37fe9b89dc
Merge pull request #12055 from leizor/leizor/prometheus/issues/12009
Adjust samplesPerChunk from 120 to 220
2023-05-10 14:45:12 +02:00
François Gouteroux b1bab7bc54 feat(promtool): add push metrics command
Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>
2023-04-27 14:49:38 +02:00
Jeanette Tan a064ec6fc1 Fix lint 2023-04-26 22:22:31 +08:00