Commit graph

8965 commits

Author SHA1 Message Date
Brian Brazil 3f8e51738c
More granular locking for scrapeLoop. (#8104)
Don't lock for all of Sync/stop/reload as that holds up /metrics and the
UI when they want a list of active/dropped targets. Instead take
advantage of the fact that Sync/stop/reload cannot be called
concurrently by the scrape Manager and lock just on the targets
themselves.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-10-26 14:46:20 +00:00
David Leadbeater e7e60623ff
promtool: Calculate mint and maxt per test (#8096)
* promtool: Calculate mint and maxt per test

Previously a single test that used a later eval time would make all
other tests in the file share the [mint, maxt] and potentially evaluate
far more samples than needed.

Fixes: #8019

Signed-off-by: David Leadbeater <dgl@dgl.cx>
2020-10-24 12:03:55 +01:00
Piper 712477be5a
docs: fix "timeseries" -> "time series" in README.md (#8107)
Signed-off-by: Piper <piperchester@gmail.com>
2020-10-24 10:27:24 +01:00
Harkishen Singh 9599bb9b38
Update remote_write default values in docs (#8103)
Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>
2020-10-23 12:12:14 +01:00
like-inspur 29b551225b
add networking.k8s.io for ingress (#8091)
* add networking.k8s.io for ingress

level=error ts=2020-10-19T08:32:30.544Z caller=klog.go:96 component=k8s_client_runtime func=ErrorDepth msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:494: Failed to watch *v1beta1.Ingress: failed to list *v1beta1.Ingress: ingresses.networking.k8s.io is forbidden: User \"system:serviceaccount:monitoring:prometheus\" cannot list resource \"ingresses\" in API group \"networking.k8s.io\" at the cluster scope"

Signed-off-by: root <likerj@inspur.com>

* Update rbac-setup.yml

Signed-off-by: root <likerj@inspur.com>
2020-10-22 15:08:12 -06:00
James Ranson 1cffda5de7
react updates for pathPrefix (#7979)
* dynamically determine path prefix

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* minor changes per PR review

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* use Context for apiPath and pathPrefix

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* remove unhandled "/version" path

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* only process index once instead of on every req

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* remove unneeded tag fragment

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* switch api path to const

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* revert

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* update tests

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* linter updates

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* simplify

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* updates per peer review

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
2020-10-22 17:22:32 +02:00
Victor Araujo 74775d7324
Add major version tag (#8026)
When the latest tag is added, also add a tag for the major version
when the version tag matches the latest release defined in VERSION.

Signed-off-by: Victor Araujo <vear91@gmail.com>
2020-10-22 17:19:44 +02:00
Julien Pivotto 4e5b1722b3
Move away from testutil, refactor imports (#8087)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-22 11:00:08 +02:00
Arthur Silva Sens c5a832b394
Close resources after failing to startup TSDB (#8031)
* Close resources after failing to startup TSDB

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Return close error instead of logging

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Change named return's name

Signed-off-by: arthursens <arthursens2005@gmail.com>
2020-10-21 20:38:28 +05:30
Brian Brazil fdf1c29224
Fix metric description of prometheus_tsdb_symbol_table_size_bytes. (#8080)
This is how much memory we use to load in the on-disk
symbol tables, not the size of the tables themselves.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-10-21 14:35:40 +01:00
Ganesh Vernekar ce4b3ac282
Simplify TestHeadReadWriter_Truncate (#7437)
* Simplify TestHeadReadWriter_Truncate

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-10-21 18:41:39 +05:30
Ganesh Vernekar 2624d827fa
Read repair empty last file in chunks_head (#8061)
* Read repair empty file in chunks_head

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor and introduce repairLastChunkFile

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Attempt windows test fix

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-10-21 18:27:13 +05:30
Frederic Branczyk 3b08b2dabe
Merge pull request #8083 from metalmatze/mixin-jsonnet-absolute-import-paths
Use absolute jsonnet import paths
2020-10-21 11:39:23 +02:00
Ricky Pai fa8722be4a
add link to circleci config in README (#8089)
Signed-off-by: Ricky Pai <rickyp999+github@gmail.com>
2020-10-21 09:37:18 +02:00
Matthias Loibl 13ba013a24
Use absolute jsonnet import paths
This should be the way forward when importing libraries in jsonnet. It's
closer to how Go imports look and makes it more obvious where packages
live.

This is not breaking anything, as the old imports were already symlinks
to the now directly used directories.

Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>
2020-10-20 11:42:30 +02:00
Bartlomiej Plotka 2fe1e9fa93
Create a checkpoint only at the end of Compact call (#8067)
* Create a checkpoint only at the end of Compact call

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix Bartek's offline reviews

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Introduce TruncateInMemory and TruncateWAL

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Small enhancements and test fixing attempts

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add TestOneCheckpointPerCompactCall

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Don't truncate WAL on block compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Simplified the algo.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Better protection around calling truncateWAL, truncate WAL on Head compaction error

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-10-19 20:57:08 +05:30
Ganesh Vernekar d9cd913219
Add links to TSDB blog post series in tsdb/README.md (#8078)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-10-19 17:17:10 +05:30
Julien Pivotto 59733b1a26
TSDB: use blocks instead of db.blocks in condition (#8068)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-19 16:51:54 +05:30
Ganesh Vernekar 5de7d15879
Consistent naming and better names for fields in TSDB format docs (#8077)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-10-19 14:27:19 +05:30
jessicagreben 36ac0b68f1 merge master, fix conflicts 2020-10-17 08:20:21 -07:00
Julien Pivotto 8c9850c2bb
Remote: Do not collect non-initialized timestamp metrics (#8060)
* Remote: Do not collect non-initialized timestamp metrics

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-15 23:53:59 +02:00
Frederic Branczyk 0a7fdd3b76
Merge pull request #8059 from brancz/cut-2.22.0
*: Cut v2.22.0
2020-10-15 13:07:37 +02:00
Bartlomiej Plotka 9981b3f3ee
labels: Reduce allocated memory and latency of Hash method; Added tests. (#8025)
* labels: Reduce allocated memory by Hash method in edge cases; Added tests.

Old:
/tmp/___BenchmarkLabels_Hash_in_github_com_prometheus_prometheus_pkg_labels -test.v -test.bench ^\QBenchmarkLabels_Hash\E$ -test.run ^$
goos: linux
goarch: amd64
pkg: github.com/prometheus/prometheus/pkg/labels
BenchmarkLabels_Hash
BenchmarkLabels_Hash/typical_labels_under_1KB
BenchmarkLabels_Hash/typical_labels_under_1KB-12         	 5366161	       259 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabels_Hash/bigger_labels_over_1KB
BenchmarkLabels_Hash/bigger_labels_over_1KB-12           	 1700371	       767 ns/op	    2048 B/op	       1 allocs/op
BenchmarkLabels_Hash/extremely_large_label_value_10MB
BenchmarkLabels_Hash/extremely_large_label_value_10MB-12 	     356	   3743115 ns/op	10523442 B/op	       1 allocs/op
PASS

New:
/tmp/___BenchmarkLabels_Hash_in_github_com_prometheus_prometheus_pkg_labels -test.v -test.bench ^\QBenchmarkLabels_Hash\E$ -test.run ^$
goos: linux
goarch: amd64
pkg: github.com/prometheus/prometheus/pkg/labels
BenchmarkLabels_Hash
BenchmarkLabels_Hash/typical_labels_under_1KB
BenchmarkLabels_Hash/typical_labels_under_1KB-12         	 4758883	       259 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabels_Hash/bigger_labels_over_1KB
BenchmarkLabels_Hash/bigger_labels_over_1KB-12           	 3324492	       357 ns/op	      80 B/op	       1 allocs/op
BenchmarkLabels_Hash/extremely_large_label_value_10MB
BenchmarkLabels_Hash/extremely_large_label_value_10MB-12 	    1087	   1083949 ns/op	    9734 B/op	       1 allocs/op
PASS

Process finished with exit code 0

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed Kemal's comment.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Moved to v2 xxhash for improvements.

New:

/tmp/___BenchmarkLabels_Hash_in_github_com_prometheus_prometheus_pkg_labels -test.v -test.bench ^\QBenchmarkLabels_Hash\E$ -test.run ^$ -test.benchtime 10s
goos: linux
goarch: amd64
pkg: github.com/prometheus/prometheus/pkg/labels
BenchmarkLabels_Hash
BenchmarkLabels_Hash/typical_labels_under_1KB
BenchmarkLabels_Hash/typical_labels_under_1KB-12         	53447894	       221 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabels_Hash/bigger_labels_over_1KB
BenchmarkLabels_Hash/bigger_labels_over_1KB-12           	42341754	       326 ns/op	       0 B/op	       0 allocs/op
BenchmarkLabels_Hash/extremely_large_label_value_10MB
BenchmarkLabels_Hash/extremely_large_label_value_10MB-12 	   10000	   1248546 ns/op	       0 B/op	       0 allocs/op
PASS

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Removed old xxhash package.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Pined minor version.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-10-15 11:31:28 +01:00
Björn Rabenstein d49f267f76
Merge pull request #8054 from simonpasquier/improve-not-ingesting-samples-alert
documentation/prometheus-mixin: improve PrometheusNotIngestingSamples
2020-10-15 12:29:39 +02:00
Simon Pasquier f381d8a9bd documentation/prometheus-mixin: improve PrometheusNotIngestingSamples
The alert shouldn't fire when there's no target and no rule configured.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2020-10-15 11:13:17 +02:00
Tobias Schmidt 77c1da8f9d
Reflect automatic docs publishing in the release process (#8056)
No more need to do anything manually for release docs.

Signed-off-by: Tobias Schmidt <tobidt@gmail.com>
2020-10-15 11:06:42 +02:00
Ganesh Vernekar 4ed3ec7b46
More info in mmap error message (#8058)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-10-15 12:58:13 +05:30
Julien Pivotto 4596abee4d
Mixin: Ignore unset remote write timestamp (#8046)
* Mixin: Ignore unset remote write timestamp

This pull request ignores the zero value of highest_sent_timestamp_seconds
in Highest Timestamp In vs. Highest Timestamp Sent which just show that
remote write has not been successful yet.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-15 09:15:59 +02:00
Frederic Branczyk e1406c9406
*: Cut v2.22.0
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
2020-10-15 07:50:33 +02:00
frbimo 260cd84d8f
log on empty head compaction result (#8037)
Signed-off-by: frbimo <fr.bimo@gmail.com>
2020-10-14 18:59:59 +05:30
Gayathri Venkatesh 73e2ce1bd6
Do not ignore reload errors in compactHead (#8051)
* Modified unknownRefs to unknownRefs.Load()

Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com>

* Modified db.go

Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com>

* Revert "Modified unknownRefs to unknownRefs.Load()"

This reverts commit 79caf595fa9b9193878dc0dd9dec17d58851ae42.

Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com>

* Made changes to reload error in db.go

Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com>
2020-10-14 15:05:24 +05:30
Goutham Veeramachaneni 1cc02930b6
Implement series API query hint to not hit chunks (#8050)
Fixes https://github.com/prometheus/prometheus/issues/5547

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2020-10-14 14:36:17 +05:30
Ling Jin 9145200842
tsdb: fix unkown ref in log (#8048)
Signed-off-by: JinLingChristopher <jinl1037@hotmail.com>
2020-10-13 20:03:16 +05:30
Bartlomiej Plotka dee6ac274e
labels: Extra tests and comment for WithoutLabels method. (#8024)
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-10-13 08:57:53 +01:00
Björn Rabenstein 71577e45eb
Merge pull request #8044 from prometheus/beorn7/metrics
Instrumentation: Report valid configs in the respective metrics from the beginning
2020-10-12 23:32:02 +02:00
Arthur Silva Sens 4f45e201cc
Promtool tsdb list now prints block sizes (#7993)
* promtool tsdb list now prints blocks' size

Signed-off-by: arthursens <arthursens2005@gmail.com>
2020-10-12 23:15:40 +02:00
beorn7 0f3c1bf6cf Report valid configs in the respective metrics from the beginning
In #7399, an early validity check of the config was introduced to
prevent the scenario where an invalid config is only detected after a
possibly very long startup procedure. However, the respective success
metrics are not updated after the initial validation so that the
success metrics suggest an invalid config. If the startup procedure,
like replaying the WAL, really takes very long, alerts about invalid
config will trigger.

This commit sets the succes metrics after initial validation. They
will be set again after the "real" config (re-)load, but that
shouldn't be a problem. The metric now truthfully represents whenever
the config was successfully loaded, no matter if the result was then
thrown away (because it was just for validation) or actually used.

Signed-off-by: beorn7 <beorn@grafana.com>
2020-10-12 21:30:59 +02:00
Jess G 90680b092c
Add a tsdb writer interface (#7950)
* add a tsdb writer interface

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>

* use block writer in tsdbutil block creator

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>

* pass thru context

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>

* use block writer in blockutil and also in tests

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>

* fix link err

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>

* add err handling for defer

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>

* add blockwriter test

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>

* mv initHead into NewBlockWriter, update tests

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>

* use blocksize arg, make test more concise

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2020-10-12 17:04:20 +01:00
Brian Brazil cf273f21bb
Add a testcase for #8040 (#8042)
This was already fixed by #8013, but add a test case anyway
in case the regexp engine changes in future.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-10-12 12:17:29 +01:00
David Leadbeater 5393ec22cb promtool: Don't end alert tests early, in some failure situations
If an alert test had a failing test, then any other alert test interval
specified after that point would result in the test exiting early.
This made debugging some tests more difficult than needed.

Now only exit early for evaluation failures.

Signed-off-by: David Leadbeater <dgl@dgl.cx>
2020-10-09 12:59:59 +01:00
Frederic Branczyk 3240cf83f0
Merge pull request #8020 from brancz/cut-2.22.0-rc.0
*: Cut v2.22.0-rc.0
2020-10-08 15:52:19 +02:00
Frederic Branczyk be137435be
*: Cut v2.22.0-rc.0
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
2020-10-08 13:52:40 +02:00
Frederic Branczyk 8703bc88cd
Merge pull request #8027 from brancz/v2.22-go-dep-update
*: Update dependencies
2020-10-08 12:08:11 +02:00
Frederic Branczyk 21a2e8c320
storage/remote/intern_test.go: Fix linting errors
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
2020-10-08 09:50:37 +02:00
Frederic Branczyk b97a1bb7d1
web/ui/react-app/yarn.lock: Update UI dependencies
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
2020-10-08 09:32:17 +02:00
Frederic Branczyk a58489857d
*: Update go dependencies
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
2020-10-08 09:29:12 +02:00
Frederic Branczyk da3ea43242
Merge pull request #7976 from roidelapluie/tolerance
Introduce timestamp tolerance in scrapes
2020-10-08 09:21:19 +02:00
Siddhant Sinha d9dcf2c90f
Fixes #7982. Increase maxErrMsgLen value remote_write api (#8017)
Signed-off-by: Siddhant Sinha <sid.sinha94@gmail.com>
2020-10-08 00:13:09 +01:00
Julien Pivotto be5ba1a62d Fix wordings
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-07 21:44:36 +02:00
Julien Pivotto 4617d16b4b Specify the removal
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-10-07 18:32:04 +02:00