Commit graph

9157 commits

Author SHA1 Message Date
Bryan Boreham 6788760efa
Reduce memory allocation in benchmarkIterator() (#5983)
Previously it was allocating millions of chunks, all containing the
same 250 samples.  Above some ratio of CPU performance to available
memory, the benchmark cannot run.

Make 250 a const and just allocate one chunk which we iterate
repeatedly till we reach the benchmark count.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-07-26 19:36:54 +05:30
darshanime c8a2ffdb72 Add computer name to azure sd
Signed-off-by: darshanime <deathbullet@gmail.com>
2021-07-25 22:07:44 +05:30
Julien Pivotto 79d354ad2e
Merge pull request #8844 from austince/feat/discovery-xds
Add base xDS REST SD and kuma_sd implementation
2021-07-23 09:46:36 +02:00
George Brighton bc0e76c8a3
Add AZ ID label to discovered EC2 targets (#8896)
* Add AZ ID to EC2 SD

Signed-off-by: George Brighton <george@gebn.co.uk>
2021-07-23 09:42:03 +02:00
austin ce 3593b20cdb
Add documentation for kuma_sd configuration
Signed-off-by: austin ce <austin.cawley@gmail.com>
2021-07-21 12:55:02 -04:00
austin ce bbc951f50b
Add config tests for kuma SD
Signed-off-by: austin ce <austin.cawley@gmail.com>
2021-07-21 12:55:02 -04:00
austin ce 0544bdd6af
Add tests for xDS discovery
Signed-off-by: austin ce <austin.cawley@gmail.com>
2021-07-21 12:55:02 -04:00
austin ce d0ffe2ed3e
Add base xDS discovery and kuma SD implementation
Signed-off-by: austin ce <austin.cawley@gmail.com>
2021-07-21 12:55:02 -04:00
austin ce 5bdfba1d20
Extract and export GetFQDN()
Signed-off-by: austin ce <austin.cawley@gmail.com>
2021-07-21 12:55:02 -04:00
Oleg Zaytsev b1ed4a0a66
LabelNames API with matchers (#9083)
* Push the matchers for LabelNames all the way into the index.

NB This doesn't actually implement it in the index, just plumbs it through for now...

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Hack it up.  Does not work.

Signed-off-by: Tom Wilkie <tom@grafana.com>

* Revert changes I don't understand

Can't see why do we need to hold a mutex on symbols, and the purpose of
the LabelNamesFor method.

Maybe I'll need to re-add this later.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Implement LabelNamesFor

This method provides the label names that appear in the postings
provided. We do that deeper than the label values because we know
beforehand that most of the label names we'll be the same across
different postings, and we don't want to go down an up looking up the
same symbols for all different series.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Mutex on symbols should be unlocked

However, I still don't understand why do we need a mutex here.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Fix head.LabelNamesFor

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Implement mockIndex LabelNames with matchers

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Nitpick on slice initialisation

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Add tests for LabelNamesWithMatchers

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Fix the mutex mess on head.LabelValues/LabelNames

I still don't see why we need to grab that unrelated mutex, but at least
now we're grabbing it consistently

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Check error after iterating postings

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Use the error from posting when there was en error in postings

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Update storage/interface.go comment

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>

* Update tsdb/index/index.go comment

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>

* Update tsdb/index/index.go wrapped error msg

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>

* Update tsdb/index/index.go wrapped error msg

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>

* Update tsdb/index/index.go warpped error msg

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>

* Remove unneeded comment

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Add testcases for LabelNames w/matchers in api.go

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* Use t.Cleanup() instead of defer in tests

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

Co-authored-by: Tom Wilkie <tom@grafana.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-07-20 18:08:08 +05:30
Jupiter 7337ecf0d3
Log when total symbol size exceeds 2^32 bytes. (#9104)
* Compaction fails when total symbol size exceeds 2^32 bytes.
Signed-off-by: tanghengjian <1040104807@qq.com>

* Compaction fails when total symbol size exceeds 2^32 bytes.

Signed-off-by: tanghengjian <1040104807@qq.com>

* Compaction fails when total symbol size exceeds 2^32 bytes.

Signed-off-by: root <tanghengjian@oppo.com>

Co-authored-by: root <tanghengjian@oppo.com>
2021-07-20 15:51:36 +05:30
Ganesh Vernekar 59d02b5ef0
tsdb: Block Head GC till pending readers are done reading (#9081)
* tsdb: Block Head GC till pending readers are done reading

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments 2

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix the exclusiveness of maxt in WaitForPendingReadersInTimeRange

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-20 14:17:20 +05:30
Martin Disibio 1bcd13d6b5
Exemplar resize (#8974)
* Create experimental circular buffer resize method, benchmarks

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Optimize exemplar resize to only replay as many exemplars as needed

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* More comments, benchmark AddExemplar

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* optimizations

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* comment

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Slight refactor of resize benchmark + make use of resize via runtime
reloadable storage config.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Some more config related changes.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address some review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address more review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Refactor to remove usage of noopExemplarStorage and avoid race condition
when resizing from Head code.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix or add comments to clarify some of the new behaviour.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* fix potential panics related to negative exemplar buffer lengths

Signed-off-by: Callum Styan <callumstyan@gmail.com>

Co-authored-by: Callum Styan <callumstyan@gmail.com>
2021-07-20 10:22:57 +05:30
Naka Masato a1c1313b3c
fix typo in comment for scrape manager (#9094)
Signed-off-by: Masato Naka <masatonaka1989@gmail.com>
2021-07-19 15:55:13 +05:30
Arunprasad Rajkumar 83a56e22ab
docs: update unit_testing_rules to cover missing and stale samples (#9065)
Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
2021-07-19 15:46:14 +05:30
Richard Hartmann f5655c47e8
Merge pull request #9091 from prometheus/superq/cleanup_indirects
Update Go modules
2021-07-15 16:24:12 +02:00
SuperQ 9e31e0a131
Update Go modules
Update several indirect dependencies. Cleanups up some obsolete indirect
requirements.
* github.com/HdrHistogram/hdrhistogram-go@v1.1.0
* github.com/containerd/containerd@v1.5.3

Signed-off-by: SuperQ <superq@gmail.com>
2021-07-15 15:24:59 +02:00
Ben Kochie a0571449fd
Merge pull request #9087 from LeviHarrison/change-disabled-features-command-text
Update "Disabled Features" links to new name
2021-07-14 10:09:06 +02:00
Levi Harrison 3b5257d869
Changed disabled_features to feature_flags
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-07-13 22:03:51 -04:00
Julien Pivotto a789ea6447
Merge pull request #9072 from darshanime/add_stringer_to_query
Add Stringer to Query interface
2021-07-13 16:21:42 +02:00
ide-rea 14b24d15b0
update checkpoint replay status (#8898)
* Consider  wal checkpoint replay status

Signed-off-by: XiaoYu Zhang <ideoutrea@163.com>

* Fix tests failed

Signed-off-by: XiaoYu Zhang <ideoutrea@163.com>

* Update checkpoint replay status

Signed-off-by: XiaoYu Zhang <ideoutrea@163.com>
2021-07-13 15:38:07 +05:30
Ben Kochie e98b639ac7
Rename "Disabled Features" docs page (#9073)
Make the feature flags page more discoverable by naming it what it is.

Signed-off-by: SuperQ <superq@gmail.com>
2021-07-12 12:00:34 +01:00
darshanime 364c40be57 Add Stringer to Query interface
Signed-off-by: darshanime <deathbullet@gmail.com>
2021-07-11 19:23:34 +05:30
Bryan Boreham dc8f505595
tsdb: coalesce lock/unlock operations for append (#9061)
Fetch the low watermark value under the same lock as we need for the
appender, rather than releasing then re-aquiring a lock on the same
Mutex.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-07-07 18:58:20 +05:30
Joey Freeland 5d0a12855b Update discovery/gce/gce.go
Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>
Signed-off-by: Joey Freeland <joey@free.land>
2021-07-06 07:57:29 -04:00
Julien Pivotto 62598878dd
Merge pull request #9050 from colega/use-binary-literals-for-xor-encoding
Use binary literals for xor chunk encoding
2021-07-05 22:46:38 +02:00
Oleg Zaytsev 40126a8494
Use binary literals for xor chunk encoding
An opinionated cosmetic change, but since go 1.13 we have this fancy
0b.... literals so we don't need to write hex and comment the binary
value.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2021-07-05 16:39:24 +02:00
Lukas Kämmerling 263847e64a
hcloud discovery: Add new labelpresent label (#9028)
* Add new labelpresent label

Signed-off-by: Lukas Kämmerling <lukas.kaemmerling@hetzner-cloud.de>
2021-07-03 01:51:50 +02:00
Filip Petkovski 7c125aa5fb
Promtool: Add support for compaction analysis (#8940)
* Extend promtool to support compaction analysis

This commit extends the promtool tsdb analyze command to help
troubleshoot high Prometheus disk usage. The command now plots a
distribution of how full chunks are relative to the maximum capacity of
120 samples per chunk.

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>

* Update cmd/promtool/tsdb.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2021-07-02 11:08:52 +01:00
Julius Volz 179b2155d1
Fix: Use json.Unmarshal() instead of json.Decoder (#9033)
* Fix: Use json.Unmarshal() instead of json.Decoder

See https://ahmet.im/blog/golang-json-decoder-pitfalls/

json.Decoder is for JSON streams, not single JSON objects / bodies.

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Revert modifications to targetgroup parsing

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-07-02 09:38:14 +01:00
Augustin Husson f72cabb437
Add sorting and filtering to flags page (v2) (#8988)
* Add sorting and filtering to flags page

Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>

* Make filter understand

Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>

* split big state object into smaller ones

Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>

* use fuzzy match and sanitize html for search results

Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>

* use fuzzy.filter

Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>

* replace fuzzy lib by @nexucis/fuzzy + fix flags issues

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* replace fuzzy by @nexucis/fuzzy in ExpressionInput.tsx

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* remove fuzzy lib from package.json

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix flags test

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* simplify the input in the fuzzy search

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* cleanup html to be easily compatible with the dark theme

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix filtering when there is no result

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* use id to fix the test

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

Co-authored-by: Dustin Hooten <dustinhooten@gmail.com>
2021-07-01 21:15:06 +02:00
Julius Volz 441e6cd7d6
Merge release-2.28 back into main (#9035)
* Cut v2.28.0-rc.0 (#8954)

* Cut v2.28.0-rc.0

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Changelog fixup

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Address review comments

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Downgrade some features to enhancements

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Adjust release date to today

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Migrate HTTP SD docs from docs repo (#8972)

See discussion in https://github.com/prometheus/docs/pull/1975

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Cut Prometheus v2.28.0 (#8973)

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* HTTP SD: Allow charset in content type (#8981)

* Added content type regex

Signed-off-by: Levi Harrison <git@leviharrison.dev>
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* fixed disappeared target groups in http_sd #9019

Signed-off-by: servak <fservak@gmail.com>

* Add a testcase for http-sd

Signed-off-by: servak <fservak@gmail.com>

* HTTP SD: Simplify logic of disappeared targetgroups (#9026)

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Fix logging after the move to go-kit/log (#9021)

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Cut Prometheus v2.28.1 (#9034)

Signed-off-by: Julius Volz <julius.volz@gmail.com>

Co-authored-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Julien Pivotto <roidelapluie@inuits.eu>
Co-authored-by: servak <fservak@gmail.com>
2021-07-01 18:02:13 +02:00
Levi Harrison 90976e7505
Promtool: Add feature flags to unit tests (#8958)
* Added feature flag support to unit tests

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added/fixed tests

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Addressed review comments

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-30 22:43:39 +01:00
Levi Harrison 167dfa19af
React UI: Select time range with mouse drag (#8977)
* Added selection flot plugin

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added time selection

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added tests

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Removed irrelevant line in license header

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-30 10:40:43 +02:00
Levi Harrison 2314a21169 Rearranged components - this time on node-disk.html
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-29 20:06:20 -04:00
Ankit Goel d437cee73a
Move storage.tsdb.retention.size out of experimental #8728 (#9004)
* Move storage.tsdb.retention.size out of experimental #8728

Signed-off-by: Ankit Goel <ankit.goel@deliveryhero.com>
2021-06-30 01:30:11 +02:00
Joey Freeland 8017dd7242 chore: always append interface ipv4 with api interface name
Signed-off-by: Joey Freeland <joey@free.land>
2021-06-29 09:01:34 -07:00
Julien Pivotto 1a1394fc58
Merge pull request #9008 from cw9/cw9/openmetricsparsets
Return error on NaN and Inf timestamps for OpenMetrics parser
2021-06-29 17:56:49 +02:00
Levi Harrison ca1896c15b
Promtool: Validate service discovery files (#8950)
* Check SD files in promtool

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-29 17:32:59 +02:00
Steve Kuznetsov fd6c852567
promtool: backfill: allow configuring block duration (#8919)
* promtool: backfill: allow configuring block duration

When backfilling large amounts of data across long periods of time, it
may in certain circumstances be useful to use a longer block duration to
increase the efficiency and speed of the backfilling process. This patch
adds a flag --block-duration-power to allow a user to choose the power N
where the block duration is 2^(N+1)h.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* promtool: use sub-tests in backfill testing

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* backfill: add messages to tests for clarity

When someone new breaks a test, seeing "expected: false, got: true" is
really not useful. A nice message helps here.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* backfill: test long block durations

A test that uses a long block duration to write bigger blocks is added.
The check to make sure all blocks are the default duration is removed.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
2021-06-29 14:53:38 +05:30
cw9 13cd50c52e Return error on NaN and Inf timestamps for OpenMetrics parser
Signed-off-by: cw9 <chaow929@gmail.com>
2021-06-28 20:55:04 -04:00
Julius Volz 664b391573
Speed up alerts page by not rendering collapsed details (#9005)
All this is doing is wrapping the inner alert details display with a
conditional `{open && ...}`.

This already improves https://github.com/prometheus/prometheus/issues/8548 a
lot for cases where there are many individual firing/pending alert elements
under each alerting rule.

E.g. for a list of 200 rules with ~100 alert elements each, this changed the page
render time from 30 seconds to 1s.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-28 16:14:48 +02:00
Levi Harrison a680a13ddd
Rearranged components
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-28 08:44:43 -04:00
Julien Pivotto 67fae26b29
Merge pull request #8994 from darshanime/federate_dup_fix
Ask querier for sorted series in /federate
2021-06-28 12:13:05 +02:00
Levi Harrison 78d5a6d083
HTTP SD: Allow charset in content type (#8981)
* Added content type regex

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-28 08:46:53 +02:00
Julien Pivotto e2557e4920
Revert "Update sync PRs" (#8998)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-06-27 00:29:56 +02:00
Julien Pivotto 03db36114c
Merge pull request #8997 from prometheus/superq/update_sync_pr
Update sync PRs
2021-06-27 00:08:17 +02:00
SuperQ c843314a9f
Update sync PRs
Push updates to the repo sync PRs if there is already a PR open. This
allows for cumulative updates to be synced.

Signed-off-by: SuperQ <superq@gmail.com>
2021-06-26 18:42:37 +02:00
Julien Pivotto 3707619900
Merge pull request #8995 from LeviHarrison/reposync-yamllint
Add .yamllint to reposync (and don't run when not installed)
2021-06-25 14:48:35 +02:00
Levi Harrison b7d2f4e189
Only run if yamllint is installed
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-25 08:03:58 -04:00