Commit graph

8091 commits

Author SHA1 Message Date
Bartlomiej Plotka 28c5cfaf0d
tsdb: Moved code merge series and iterators to differen files; cleanup. No functional changes just move! (#7714)
I did not want to move those in previous PR to make it easier to review. Now small cleanup time for readability. (:

## Changes

* Merge series goes to `storage/merge.go` leaving `fanout.go` for just fanout code.
* Moved `fanout test` code from weird separate package to storage.
* Unskiped one test: TestFanout_SelectSorted/chunk_querier
* Moved block series set codes responsible for querying blocks to `querier.go` from `compact.go`



Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-08-03 11:32:56 +01:00
Guangwen Feng e6f2b8ad60
Fix golint issue in labels.go (#7730)
Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
2020-08-03 11:25:23 +01:00
johncming ac677ed8b3
promql: delete redundant return value. (#7721)
Signed-off-by: johncming <johncming@yahoo.com>
2020-08-03 10:45:53 +01:00
johncming 01d096567b
discovery: simplify code. (#7725)
Signed-off-by: johncming <johncming@yahoo.com>
2020-08-03 10:29:59 +01:00
Julien Pivotto 9848e77678
storage/remote: remove unused code now that tsdb implements ChunkQuerier (#7715)
* storage/remote: remove unused code now that tsdb implements ChunkQuerier

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* encodeChunks is unused too

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-08-03 10:13:36 +02:00
Julien Pivotto 3a7120bc07 Consul: Reduce WatchTimeout to 2m and set it as timeout for requests
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-08-03 00:42:55 +02:00
johncming 31929b83d5
promql: use explicit type declare instead of string. (#7716)
Signed-off-by: johncming <johncming@yahoo.com>
2020-08-02 09:57:38 +01:00
johncming 1c1b394e5e
promql: Swap order of parseBrokenJSON. (#7718)
Signed-off-by: johncming <johncming@yahoo.com>
2020-08-02 09:48:57 +01:00
johncming 5578c96307
scrape: fix typo. (#7712)
Signed-off-by: johncming <johncming@yahoo.com>
2020-08-01 09:56:21 +01:00
Julien Pivotto 32c2d897ec
Storage: add chunks_head and fix links (#7693)
The links fix makes the links point to the current version of the files,
e.g. the relase branch.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-31 19:17:16 +02:00
Julien Pivotto 7b5507ce4b
Scrape: defer report (#7700)
When I started wotking on target_limit, scrapeAndReport did not exist
yet. Then I simply rebased my work without thinking.

It appears that there is a lot that can be inline if I defer() the
report.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-31 19:11:08 +02:00
Julien Pivotto 30e079bbd5
TSDB: Fix master tests (#7705)
Now appenders take a context.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-31 17:33:54 +02:00
Bartlomiej Plotka e6d7cc5fa4
tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069)
* tsdb: Added ChunkQueryable implementations to db; unified compactor, querier and fanout block iterating.

Chained to https://github.com/prometheus/prometheus/pull/7059

* NewMerge(Chunk)Querier now takies multiple primaries allowing tsdb DB code to use it.
* Added single SeriesEntry / ChunkEntry for all series implementations.
* Unified all vertical, and non vertical for compact and querying to single
merge series / chunk sets by reusing VerticalSeriesMergeFunc for overlapping algorithm (same logic as before)
* Added block (Base/Chunk/)Querier for block querying. We then use populateAndTomb(Base/Chunk/) to iterate over chunks or samples.
* Refactored endpoint tests and querier tests to include subtests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed comments from Brian and Beorn.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed snapshot test and added chunk iterator support for DBReadOnly.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed race when iterating over Ats first.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed populate block tests.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed endpoints test.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed test.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added test & fixed case of head open chunk.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed DBReadOnly tests and bug producing 1 sample chunks.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added cases for partial block overlap for multiple full chunks.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added extra tests for chunk meta after compaction.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed small vertical merge bug and added more tests for that.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-07-31 16:03:02 +01:00
Frederic Branczyk db7c5e204a
Merge pull request #7660 from annanay25/appender-context
Add context to storage.Appendable interface
2020-07-31 11:40:43 +02:00
Annanay ec562f152b Merge branch 'master' into appender-context
Signed-off-by: Annanay <annanayagarwal@gmail.com>
2020-07-31 13:03:56 +05:30
johncming 3c5ff27959
Update http header User-Agent in k8s and swarm discovery. (#7614)
* discovery: update k8s and swarm user agent.

Signed-off-by: johncming <johncming@yahoo.com>
2020-07-31 00:18:38 +02:00
Julien Pivotto f482c7bdd7
Add per scrape-config targets limit (#7554)
* Add per scrape-config targets limit

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-30 14:20:24 +02:00
Annanay 48b9afd14b Address comments
Signed-off-by: Annanay <annanayagarwal@gmail.com>
2020-07-30 17:25:51 +05:30
Annanay 0b4d448d29 Fix tests
Signed-off-by: Annanay <annanayagarwal@gmail.com>
2020-07-30 17:18:47 +05:30
Annanay 263d2aa5f5 Fix failing tests
Signed-off-by: Annanay <annanayagarwal@gmail.com>
2020-07-30 17:06:56 +05:30
Annanay 9bba8a6eae Merge branch 'master' into appender-context
Signed-off-by: Annanay <annanayagarwal@gmail.com>
2020-07-30 16:43:18 +05:30
Annanay 89129cd39a Address comments
Signed-off-by: Annanay <annanayagarwal@gmail.com>
2020-07-30 16:41:13 +05:30
Julien Pivotto 01e3bfcd1a
Add warnings about NFS (#7691)
* Add warnings about NFS

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-30 11:22:44 +02:00
Javier Palomo Almena b58a613443
Replace sync/atomic with uber-go/atomic (#7683)
* storage: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* tsdb: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* web: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* notifier: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* cmd: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* scripts: Verify that we are not using restricted packages

It checks that we are not directly importing 'sync/atomic'.

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* Reorganise imports in blocks

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* notifier/test: Apply PR suggestions

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* storage/remote: avoid storing references on newEntry

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* Revert "scripts: Verify that we are not using restricted packages"

This reverts commit 278d32748e.

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* web: Group imports accordingly

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
2020-07-30 13:15:42 +05:30
Julien Pivotto 3e0ec3f8e1 Minor change
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-29 22:14:23 +02:00
Julien Pivotto 924e7239b7
Docker Swarm SD: Support tasks and service without published ports (#7686)
* Support tasks and service without published ports

Mimics k8s discovery behaviour.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-29 20:56:30 +02:00
Joe Elliott 04b028f1e6
Exports recoverable error (#7689)
Signed-off-by: Joe Elliott <number101010@gmail.com>
2020-07-29 11:08:25 -06:00
Ganesh Vernekar 48e3473b15
Add Bartek as maintainer for TSDB (#7690)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-07-29 20:25:14 +05:30
Julien Pivotto 88bdb13c55
DNS SD: add srv record target and port meta labels (#7678)
* DNS SD: add srv record target and port meta labels

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-28 22:09:01 +02:00
Javier Palomo Almena 348ff4285f
tsdb: Replace sync/atomic with uber-go/atomic in tsdb (#7659)
* tsdb/chunks: Replace sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* tsdb/heaad: Replace sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* vendor: Make go.uber.org/atomic a direct dependency

There is no modifications to go.sum and vendor/ because
it was already vendored.

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* tsdb: Remove comments referring to the sync/atomic alignment bug

Related: https://golang.org/pkg/sync/atomic/#pkg-note-BUG

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
2020-07-28 10:12:42 +05:30
Julien Pivotto 6f296594a8
Update go deps + use exclude directive (#7624)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-27 11:08:38 +02:00
Julien Pivotto e76c436e9c
Goleak in discoveries, scrape, rules (#7662)
* Add go leak tests for discoveries with goroutines

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Add go leak tests in rules

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Add go leak tests in scrape tests

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-27 09:38:08 +01:00
Julien Pivotto e07415227a
discovery: check for nil triton_sd_config (#7671)
* discovery: check for nil triton_sd_config

Note: this was discovered thanks to the added test.
The test is pretty low-level but also effective.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-26 01:20:40 +02:00
Julius Volz 9da59c856d
Improve scalar literal format documentation (#7651)
* Improve scalar literal format documentation

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Address review comments

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fixup

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2020-07-25 13:34:57 +02:00
Julien Pivotto 22acb87e09
refactoring: make sure that query_duration_seconds metrics are the same (#7668)
* refactoring: make sure that query_duration_seconds are the same

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-25 11:55:59 +02:00
Owen Diehl 00b7bdb1b6
parser.{Expr,Statement} publicly implementable (#7639)
* parser.{Expr,Statement} publicly implementable

Signed-off-by: Owen Diehl <ow.diehl@gmail.com>
2020-07-25 09:05:58 +01:00
Julien Pivotto e17d18a043
docs: Fix type of the histogram_quantile function (#7666)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-25 00:21:50 +02:00
Annanay f40e4579b7 gofmt
Signed-off-by: Annanay <annanayagarwal@gmail.com>
2020-07-24 20:40:19 +05:30
Frederic Branczyk 9f9fb1ab33
documentation: Adapt Kubernetes RBAC to use metrics roles (#3661) 2020-07-24 16:36:56 +02:00
Annanay 7f98a744e5 Add context to Appender interface
Signed-off-by: Annanay <annanayagarwal@gmail.com>
2020-07-24 19:40:51 +05:30
chinhnc e05c19da5d
Display block duration in promtool list blocks command (#7653)
* Update tsdb.go

Added DURATION column to `tsdb list` command

Signed-off-by: soup <chicknsoupuds@gmail.com>

* Use time.Duration instead of hardcoded hour

Signed-off-by: soup <chicknsoupuds@gmail.com>
2020-07-24 19:01:20 +05:30
Guangwen Feng 6b7ac2ac1b
Add unit test case to improve test coverage for matcher.go (#7658)
Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>
2020-07-24 11:21:42 +01:00
Julien Pivotto 93e9c010f3
Add more Go leak tests (#7652)
* Implement go leak test for promql

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Implement go leak test for Consul SD

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Implement go leak test in discovery manager

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-24 10:10:20 +01:00
Bartlomiej Plotka 841b13641c
promql: Refactored subquery hint tests and added todos. (#7636)
* promql: Refactorer subquery hint tests and added todos.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* fmt.


Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixes.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-07-23 23:05:43 +01:00
Ben Ye 50c261502e
add tsdb cmds into promtool (#6088)
Signed-off-by: yeya24 <yb532204897@gmail.com>

update tsdb cli in makefile and promu

Signed-off-by: yeya24 <yb532204897@gmail.com>

remove building tsdb bin

Signed-off-by: yeya24 <yb532204897@gmail.com>

remove useless func

Signed-off-by: yeya24 <yb532204897@gmail.com>

refactor analyzeBlock

Signed-off-by: yeya24 <yb532204897@gmail.com>

Fix Makefile

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2020-07-23 19:35:50 +01:00
johncming 9801f52b0a
tsdb/chunks: fix bug of data race(#7643). (#7646)
Signed-off-by: johncming <johncming@yahoo.com>
2020-07-23 18:05:19 +05:30
Julien Pivotto 610b622520
Merge pull request #7644 from prometheus/release-2.20
Merge release 2.20 into master
2020-07-23 09:47:47 +02:00
Björn Rabenstein e5a06b4835
Merge pull request #7641 from prometheus/beorn7/release
Cut final v2.20.0
2020-07-22 20:45:56 +02:00
beorn7 c7013c0d4e Cut final v2.20.0
Signed-off-by: beorn7 <beorn@grafana.com>
2020-07-22 17:44:53 +02:00
Ganesh Vernekar 4a8531a64b
BlocksToDelete function in DB options (#7638)
* Optional retention filter for DB

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Specify len for the map creation

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-07-22 20:49:33 +05:30