Commit graph

95 commits

Author SHA1 Message Date
Ganesh Vernekar 095f572d4a
Sync sparsehistogram branch with main (#9189)
* Fix `kuma_sd` targetgroup reporting (#9157)

* Bundle all xDS targets into a single group

Signed-off-by: austin ce <austin.cawley@gmail.com>

* Snapshot in-memory chunks on shutdown for faster restarts (#7229)

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Rename links

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Remove Individual Data Type Caps in Per-shard Buffering for Remote Write (#8921)

* Moved everything to nPending buffer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Simplify exemplar capacity addition

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added pre-allocation

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Don't allocate if not sending exemplars

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Avoid deadlock when processing duplicate series record (#9170)

* Avoid deadlock when processing duplicate series record

`processWALSamples()` needs to be able to send on its output channel
before it can read the input channel, so reads to allow this in case the
output channel is full.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* processWALSamples: update comment

Previous text seems to relate to an earlier implementation.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Optimise WAL loading by removing extra map and caching min-time (#9160)

* BenchmarkLoadWAL: close WAL after use

So that goroutines are stopped and resources released

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* BenchmarkLoadWAL: make series IDs co-prime with #workers

Series are distributed across workers by taking the modulus of the
ID with the number of workers, so multiples of 100 are a poor choice.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* BenchmarkLoadWAL: simulate mmapped chunks

Real Prometheus cuts chunks every 120 samples, then skips those samples
when re-reading the WAL. Simulate this by creating a single mapped chunk
for each series, since the max time is all the reader looks at.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Fix comment

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Remove series map from processWALSamples()

The locks that is commented to reduce contention in are now sharded
32,000 ways, so won't be contended. Removing the map saves memory and
goes just as fast.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* loadWAL: Cache the last mmapped chunk time

So we can skip calling append() for samples it will reject.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Improvements from code review

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Full stops and capitals on comments

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Cache max time in both places mmappedChunks is updated

Including refactor to extract function `setMMappedChunks`, to reduce
code duplication.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Update head min/max time when mmapped chunks added

This ensures we have the correct values if no WAL samples are added for
that series.

Note that `mSeries.maxTime()` was always `math.MinInt64` before, since
that function doesn't consider mmapped chunks.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Split Go and React Tests (#8897)

* Added go-ci and react-ci

Co-authored-by: Julien Pivotto <roidelapluie@inuits.eu>
Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Remove search keymap from new expression editor (#9184)

Signed-off-by: Julius Volz <julius.volz@gmail.com>

Co-authored-by: Austin Cawley-Edwards <austin.cawley@gmail.com>
Co-authored-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Julien Pivotto <roidelapluie@inuits.eu>
Co-authored-by: Bryan Boreham <bjboreham@gmail.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
2021-08-11 15:43:17 +05:30
Darshan Chaudhary c4f2e9eec5
Add present_over_time (#9097)
* Add present_over_time

Signed-off-by: darshanime <deathbullet@gmail.com>

* Add tests for present_over_time

Signed-off-by: darshanime <deathbullet@gmail.com>

* Address PR comments

Signed-off-by: darshanime <deathbullet@gmail.com>

* Add documentation for present_over_time

Signed-off-by: darshanime <deathbullet@gmail.com>

* Update documentation

Signed-off-by: darshanime <deathbullet@gmail.com>

* Update documentation comment

Signed-off-by: darshanime <deathbullet@gmail.com>
2021-07-29 12:38:11 +02:00
Richard Hartmann d68d3983d8 Make clear that start/end are inclusive
Fixes: https://github.com/prometheus/prometheus/issues/9100

Signed-off-by: Richard Hartmann <richih@richih.org>
2021-07-28 14:52:02 +02:00
smedbergM 034cdb0aa4 Revert change in label_join per @RichiH
Signed-off-by: smedbergM <matthew.smedberg@gmail.com>
2021-06-15 13:55:02 -06:00
smedbergM 918c06feb9 Merge branch 'main' into label-replace-docs 2021-06-15 12:27:09 -06:00
Levi Harrison 7bc11dcb06
React UI: Add Starting Screen (#8662)
* Added walreplay API endpoint

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added starting page to react-ui

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Documented the new endpoint

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed typos

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Co-authored-by: Julius Volz <julius.volz@gmail.com>

* Removed logo

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed isResponding to isUnexpected

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed width of progress bar

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed width of progress bar

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added DB stats object

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Updated starting page to work with new fields

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 2)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 3)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (and also implementing a method this time) (pt. 4)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (and also implementing a method this time) (pt. 5)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed const to let

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 6)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Remove SetStats method

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added comma

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed api

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed to triple equals

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed data response types

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Don't return pointer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed version

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed interface issue

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed pointer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed copying lock value error

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Co-authored-by: Julius Volz <julius.volz@gmail.com>
2021-06-05 15:29:32 +01:00
Matthew Smedberg 8490273bac docs :: querying :: functions :: label_replace
Clarify documentation for label_replace() because of ambiguities
between label keys and label values.

Signed-off-by: Matthew Smedberg <matthew.smedberg@gmail.com>
2021-04-27 10:32:36 -06:00
Julien Pivotto 0414534499
Merge pull request #6947 from prometheus/document-global-labels
Add new "globalUrl" field to /api/v1/targets docs
2021-03-30 22:41:47 +02:00
Callum Styan 289ba11b79
Add circular in-memory exemplars storage (#6635)
* Add circular in-memory exemplars storage

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>

* Fix some comments, clean up exemplar metrics struct and exemplar tests.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix exemplar query api null vs empty array issue.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>
2021-03-16 15:17:45 +05:30
Julius Volz 1cba174182
Document lack of metadata deletion in deletion API (#8376)
* Document lack of metadata deletion in deletion API

See also https://github.com/prometheus/prometheus/issues/7932

I also fixed the other "NOTE" callouts to be formatted in such a way that our
docs system formats them as callout boxes.

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* WIP

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Improve wordings

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-03-15 23:09:29 +01:00
schou 65658091ed moved details of feature to disabled features.md
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-03-03 21:42:17 -05:00
schou 946dba6a7f small suggested changes
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-03-03 21:35:51 -05:00
pschou f80b52be69
Merge branch 'main' into dev_neg_offset 2021-02-23 20:52:57 -05:00
schou 71712183ac minor wording updates
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:26:35 -05:00
schou 22cd48868a adding feature flag, promql-negative-offset
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:25:56 -05:00
schou 8aa9cc3978 add blob to basics documentation about negative shift
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:25:56 -05:00
pschou aff3c702ab
promql: Add sgn, clamp and last_over_time functions (#8457)
* Add sgn, clamp and last_over_time functions

Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-20 16:34:52 +01:00
Ganesh Vernekar 86c71856e8
Add start() and end() pre-processors for @ modifier (#8425)
* Add start() and end() pre-processors for @ modifier

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix reviews

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2021-02-09 21:33:16 +05:30
Ganesh Vernekar 9199fcb8d1
'@ <timestamp>' modifier (#8121)
This commit adds `@ <timestamp>` modifier as per this design doc: https://docs.google.com/document/d/1uSbD3T2beM-iX4-Hp7V074bzBRiRNlqUdcWP6JTDQSs/edit.

An example query:

```
rate(process_cpu_seconds_total[1m]) 
  and
topk(7, rate(process_cpu_seconds_total[1h] @ 1234))
```

which ranks based on last 1h rate and w.r.t. unix timestamp 1234 but actually plots the 1m rate.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2021-01-20 16:27:39 +05:30
Ben Ye caa173d2aa
Support matchers for Labels API (#8301)
Signed-off-by: Ben Ye <yb532204897@gmail.com>
Co-authored-by: Erik Klockare <eklockare@gmail.com>
2020-12-22 11:02:19 +00:00
Justin Toh 5f103aaecc
docs: Fix "time duration" link (#8209)
Signed-off-by: Justin Toh <tohjustin@hotmail.com>
2020-11-20 12:25:45 +00:00
Dustin Hooten 916dbd4c8a
React UI: Implement missing TSDB head stats section (#7876)
* React UI: Implement missing TSDB head stats section

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* Add break

Signed-off-by: Dustin Hooten <dhooten@splunk.com>
2020-09-29 22:05:33 +02:00
Julien Pivotto e208afcc95
web: Remove APIv2 (#7935)
* web: Remove APIv2

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-09-15 09:30:55 +02:00
showuon dba729f6b6
Referring to configuration.md, to have a Generic placeholders definition section (#7875)
Signed-off-by: Luke Chen <showuon@gmail.com>
2020-09-01 09:37:09 +01:00
Julien Pivotto 6f9e7ff750
Drop metric name in bool comparison between two instant vectors (#7819)
* Drop metric name in bool comparison between two instant vectors

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-08-22 21:04:03 +02:00
showuon 274dce9de2
Add a link to Clean TombStone for better reference (#7830)
Signed-off-by: Luke Chen <showuon@gmail.com>
2020-08-20 12:57:01 +01:00
Julien Pivotto d867491364
Human-friendly durations in PromQL (#7713)
* Add support for user-friendly durations

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-08-04 21:12:41 +02:00
Julius Volz 9da59c856d
Improve scalar literal format documentation (#7651)
* Improve scalar literal format documentation

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Address review comments

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fixup

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2020-07-25 13:34:57 +02:00
Julien Pivotto e17d18a043
docs: Fix type of the histogram_quantile function (#7666)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-25 00:21:50 +02:00
beorn7 cf698f71e5 Add an explanation for the quantile aggregation operator
Sadly, just linking to the Histogram best practice document, as done
for `histogram_quantile`, would be confusing here because the best
practice document only deals with quantiles in the context of
Histograms and Summaries, which is very different from the context of
the `quantile` aggregator and `quantile_over_time` function, which is
already a source of a lot of confusion.

Thus, I think the least bad solution is to add a short explanation in
this section directly. There isn't even a good resource on the
internet we can link to. A lot of statisticians use φ-quantiles, but
they don't have a generally accepted name for it.

I have added the explanation after the other detailed explanations of
`count_values`, `topk` and `bottomk`. I think that fits quite nicely
into the flow.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-07-06 19:53:12 +02:00
Julien Pivotto 72425d4e3d
Add group() aggregator (#7480)
* Add group() aggregator

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-06-30 16:51:18 +02:00
B++ d6374ae1b6
Return NaN for histogram_quantile when buckets have 0 observations (#7318)
Signed-off-by: jberny <f.bernardi89@gmail.com>
2020-06-01 09:40:39 +01:00
Jess G fdc49fae5b
Added time range parameters to labelNames API (#7288)
* add time range params to labelNames api

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* evaluate min/max time range when reading labels from the head

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* add time range params to labelValues api

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* fix test, add docs

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* add a test for head min max range

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* fix test to match comment

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* address CR comments

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* combine vars only used once

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* add time range params to labelNames api

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* evaluate min/max time range when reading labels from the head

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* add time range params to labelValues api

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* fix test, add docs

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* add a test for head min max range

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* fix test to match comment

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* address CR comments

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* combine vars only used once

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* fix test

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* restart ci

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>

* use range expectedLabelNames instead of range actualLabelNames in test

Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
2020-05-30 13:50:09 +01:00
Harold Dost 18d45e564b
Documentation: Update example expressions to follow convention. (#7195)
Based out of conversation on #7193

Signed-off-by: Harold Dost <h.dost@criteo.com>
2020-05-02 12:52:24 +01:00
Julius Volz a3270a3a0f Add new "globalUrl" field to /api/v1/targets docs
See https://github.com/prometheus/prometheus/pull/6794

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2020-03-09 14:07:30 +01:00
Brian Brazil 44ad28dd5e
PromQL: Allow more keywords as metric names (#6933)
* Allow more keywords as metric names
* Add documentation about forbidden keywords

Signed-off-by: Tobias Guggenmos <tobias.guggenmos@uni-ulm.de>
2020-03-05 13:20:53 +00:00
Joaquín Fernández Campo 088614a17f Added the version to tsdb stats api methods (#6656)
* Added the version to tsdb stats api methods
* Updated changelog.md with references to the status page PRs

Signed-off-by: Joaquin Fernandez Campo <jfcampo@gmail.com>
2020-01-19 09:54:39 +00:00
Clayton O'Neill 7e76edc300
Fix curl args for data-urlencode
The curl argument `--data-urlencode` doesn't accept equal signs between
the arg and the value, but needs whitespace instead.

Signed-off-by: Clayton O'Neill <claytono@github.com>
2020-01-14 07:10:16 -05:00
Julien Pivotto e0afec906f add absent_over_time (#6490)
* Implement absent_over_time

Fixes #2882

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-03 15:26:12 +00:00
Tobias Guggenmos 1e0cd28bf6 PromQL: Support trailing commas in grouping opts (#6480)
* PromQL: Support trailing commas in grouping opts

Fixes #6470

Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2019-12-20 11:28:56 +00:00
Julien Pivotto 06b4744793 docs: fix absent documentation (#6492)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-12-20 07:33:08 +00:00
Tom Janson 4f1cfd6cfe remove redundant/accidental line in query doc (#6482)
Signed-off-by: Tom Janson <tom.janson@lancom.de>
2019-12-18 17:31:28 +00:00
Björn Rabenstein af2257b100
Merge pull request #6442 from gotjosh/single-metric-api
api: provide metadata for a specified metric
2019-12-11 18:21:16 +01:00
Simon Pasquier 941568839a
docs: update /api/v1/rules documentation (#6449)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-12-11 17:06:17 +01:00
gotjosh e1f2da74a3 api: provide metadata for a single metric
This adds support for a new query param on the new `api/v1/metadata`
endpoint that provides metadata for a specified metric via the V1 API.

It collapses metadata that is equal across all targets, and aggregates
under the same metric name the ones that differ.

Signed-off-by: gotjosh <josue@grafana.com>
2019-12-10 21:17:49 +00:00
gotjosh 0a0a228db3 api: provide per metric metadata (#6420)
* api: provide per metric metadata

This adds a new endpoint that provides per metric metadata via the V1 API.

It collapses metadata that is equal across all targets, and aggregates under the same metric name the ones that differ.

* Allow tests to be asserted on response length

Some tests e.g. limit on API responses, don't require an assertion on
equality.

This allows us to assert against response length instead of
equality.

Signed-off-by: gotjosh <josue@grafana.com>
2019-12-10 14:56:16 +00:00
Julien Pivotto 2a190d1168 Fix small typo in docs (#6345)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-11-20 10:12:47 +01:00
Sharad Gaur a85e7aac0e Adding TSDB Stats Page in React UI (#6281)
Signed-off-by: Sharad Gaur <sgaur@splunk.com>
2019-11-12 11:15:20 +01:00
Dustin Hooten ca60bf298c React UI: Implement /targets page (#6276)
* Add LastScrapeDuration to targets endpoint

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* Add Scrape job name to targets endpoint

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* Implement the /targets page in react

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* Add state query param to targets endpoint

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* Use state filter in api call

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* api feedback

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* pr feedback frontend

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* Implement and use localstorage hook

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* PR feedback

Signed-off-by: Dustin Hooten <dhooten@splunk.com>
2019-11-11 22:42:24 +01:00
Julius Volz f7446778f3
Document new status API endpoints (#6262)
* Document new status API endpoints

These were added in https://github.com/prometheus/prometheus/pull/6243
for the React UI.

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Update new-in notices

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-11-02 18:18:32 +01:00