Commit graph

217 commits

Author SHA1 Message Date
Björn Rabenstein 3b3eaf3496
Merge pull request #5787 from cstyan/reshard-max-logging
Add metrics for max/min/desired shards to queue manager.
2019-09-09 22:32:54 +02:00
Callum Styan a98599bea8 Update remote write max shards alert; properly template/query for max
shards in description.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-09-09 12:01:11 -07:00
李国忠 d89e783217 [bugfix] custom SD: when ip out of order, reflect.deepEqual can not correctly identify whether there is a change (#5856)
* [bugfix] custom SD: when ip out of order, reflect.deepEqual can not correctly identify whether there is a change

Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>

* [format] makefile:Makefile.common:116: common-style

Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>

* [bugfix] custom sd: simonpasquier comment,It would be simpler to sort the targets alphabetically and keep reflect.DeepEqual.

Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>

* [bugfix]custom SD:fix sort

Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>

* [bugfix] custom SD : adapter.go need an empty line after "sort"

Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>

* [bugfix]custom SD:test sign-off

Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>

* [bugfix]custom SD: fix adaper_test.go

Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>
2019-08-22 11:49:45 +02:00
Ganesh Vernekar 5ecef3542d
Cleanup after merging tsdb into prometheus
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-08-13 14:04:14 +05:30
Callum Styan 3b75614892 Add a warning alert, since the remote write behind alert will probably
already be going off, about desired shards being higher than max shards.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-08-08 06:45:46 -07:00
Simon Pasquier dd174963a2 prometheus-mixin: remove PrometheusTSDBWALCorruptions
The counter is only increased when tsdb.Open() is called which
Prometheus does only once in its lifetime (when it initializes). If the
corruption can't be recovered, tsdb.Open() returns an error and
Prometheus exits. Hence the metric is either 0 (no corruption) or 1
(corruption detected and repaired). If the latter, the alert isn't
actionable and the only way to resolve it is to restart Prometheus which
would reset the counter.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-08-06 14:36:56 +02:00
Vadym Martsynovskyy a9970a47ef Fix incorrect examples in docs
Signed-off-by: Vadym Martsynovskyy <vmartsynovskyy@gmail.com>
2019-08-04 16:42:42 -07:00
Matthias Loibl 20d12ff1c7
Fix prometheus-mixin dashboards to use grafanaDashboards
Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>
2019-07-11 15:40:26 +02:00
beorn7 4825585834 Tweak tenses
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-28 17:37:49 +02:00
beorn7 9a2177949d Protect gauge-based alerts against failed scrapes
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-28 16:46:19 +02:00
beorn7 52707535b8 Remove/improve unused variables and weird doc comments
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-28 15:41:31 +02:00
beorn7 7a25a2586d Sync with alerts from kube-prometheus
While doing so, re-introduce the summary/description
annotations. Also, add a few more rules and tweak a few of the
existing ones.

Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-27 23:50:26 +02:00
beorn7 ded0705bdc Update remote repo for grafana-builder dependency
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-27 14:39:38 +02:00
beorn7 1336a28848 Use a config variable for the Prometheus name
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-27 14:34:11 +02:00
beorn7 613cb5430c Add a "work in progress" disclaimer.
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-26 23:24:22 +02:00
beorn7 e34af6d4d3 Address various comments from the review
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-26 23:22:16 +02:00
beorn7 23c03207e9 Fixed indentation
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-26 20:31:05 +02:00
beorn7 d5845ad05b Fix formatting
This is the outcome of `make fmt`.

Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-26 16:23:25 +02:00
beorn7 d45e8a0f61 Adjust to jsonnet v0.13
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-26 16:22:21 +02:00
beorn7 5c04ef3935 Make README.md immediately useful
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-26 16:12:59 +02:00
beorn7 ddfabda152 Add Makefile and suitable jsonnet files
This makes the mixins usable as abvertised.

Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-26 15:30:55 +02:00
beorn7 e943803a3c Add .gitignore file
Signed-off-by: beorn7 <beorn@grafana.com>
2019-06-26 15:22:23 +02:00
Björn Rabenstein 498d31e178
Merge pull request #5681 from prometheus/beorn7/mixin
Merge master into mixin
2019-06-19 23:17:41 +02:00
Callum Styan a5762f3681 Add dashboard for remote write to prometheus-mixin.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-06-17 15:02:42 -07:00
beorn7 5639aaf0a4 Merge branch 'master' into mixin 2019-06-17 13:07:11 +02:00
Romain Baugue 95193fa027 Exhaust every request body before closing it (#5166) (#5479)
From the documentation:
> The default HTTP client's Transport may not
> reuse HTTP/1.x "keep-alive" TCP connections if the Body is
> not read to completion and closed.

This effectively enable keep-alive for the fixed requests.

Signed-off-by: Romain Baugue <romain.baugue@elwinar.com>
2019-04-18 09:50:37 +01:00
qinng cc75c27580 Fix multiple response.WriteHeader calls error in remote read adapter (#5159)
* fix multiple response.WriteHeader calls in remote read adapter
* remove useless return

Signed-off-by: qinng <guoruyi1@xiaomi.com>
2019-04-10 13:25:35 +01:00
Tariq Ibrahim 8fdfa8abea refine error handling in prometheus (#5388)
i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors.
ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives.
iii) Does away with the use of fmt package for errors in favour of pkg/errors

Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-26 00:01:12 +01:00
Tom Wilkie 38a9bbbec2 Loosen off PrometheusRemoteWriteBehind alert.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-03-04 12:47:24 +00:00
Tom Wilkie b615069289 Update metric names.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-03-01 07:39:48 -08:00
LongKB 23480bef43 Remove the duplicated words (#5251)
Although it is spelling mistakes, it might make an affects while reading.

Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>
2019-02-22 14:32:34 +01:00
Nguyen Hai Truong 5fbda4c9d7 Secure http links (#5244)
Fix http link to https link for secure, modify http to https
in the links of project. Have some http links doesn't
redirect into https.

Co-Authored-By: Nguyen Van Trung trungnv@vn.fujitsu.com
Signed-off-by: Nguyen Hai Truong <truongnh@vn.fujitsu.com>
2019-02-21 10:48:47 +01:00
Kim Bao Long 94f5352951 Trivial fix: Fix some typos in comments
Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>
2019-02-21 09:07:49 +07:00
Tom Wilkie e248ffb220 Add alert for WAL remote write falling behind.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2019-02-12 15:22:58 +00:00
Callum Styan 5358f76c5c update remote write path proto so that Labels/Timeseries can't be nil (#4957)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-01-15 19:13:39 +00:00
Simon Pasquier 375ad1185c
*: bump gRPC dependencies (#5075)
* *: bump gRPC dependencies

This change updates the gRPC dependencies to more recent versions:

* github.com/gogo/protobuf => v1.2.0
* github.com/grpc-ecosystem/grpc-gateway => v1.6.3
* google.golang.org/grpc => v1.17.0

In addition scripts/genproto.sh leverages Go modules information instead of
hardcoding SHA1 commits. This ensures that the code is generated from
the exact same sources.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Run 'make proto' in CI

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Revert tabs -> spaces change

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix 'make proto' step

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* 'go get' grpc/protobuf dependencies

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Prepopulate cache with go mod download

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-15 15:32:05 +01:00
Simon Pasquier f678e27eb6
*: use latest release of staticcheck (#5057)
* *: use latest release of staticcheck

It also fixes a couple of things in the code flagged by the additional
checks.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Use official release of staticcheck

Also run 'go list' before staticcheck to avoid failures when downloading packages.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-04 14:47:38 +01:00
Thomas J. Fox 11a93b2b37 fix link in docs/examples/k8s-rabbitmq readme (#4967)
Signed-off-by: Thomas J. Fox <thomasjfox1@gmail.com>
2018-12-18 11:53:55 +01:00
Yaqiang Wang 8b85d876f2 fix file_sd never stop update 'custom_sd.json' file in adapter.go (#4567)
Signed-off-by: wangyaqiang1 <wangyaqiang1@jd.com>
2018-11-30 10:32:17 +01:00
Ben Kochie c6399296dc
Fix spelling/typos (#4921)
* Fix spelling/typos

Fix spelling/typos reported by codespell/misspell.
* UK -> US spelling changes.

Signed-off-by: Ben Kochie <superq@gmail.com>
2018-11-27 17:44:29 +01:00
Alex Yu 5dcce32ef8 update promlog to latest version (#4876)
* update promlog to latest version

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* Update api tests, fix main setup

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* tidy go.sum

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* revendor prometheus/common

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* only initialize config; use kingpin for remote_storage_adapter

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* actually parse the flags

Signed-off-by: Alex Yu <yu.alex96@gmail.com>

* clean up imports

Signed-off-by: Alex Yu <yu.alex96@gmail.com>
2018-11-23 14:22:40 +01:00
Tom Wilkie 638204c775 Typo
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-19 12:23:42 +00:00
Simon Pasquier ed19373a78
*: remove use of golang.org/x/net/context (#4869)
* *: remove use of golang.org/x/net/context

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* scrape: fix TestTargetScrapeScrapeCancel

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-19 12:31:16 +01:00
Tom Wilkie 8f42192e52 Add Prometheus alerts from kube-prometheus, remove the alertmanager alerts.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-19 11:22:55 +00:00
Tom Wilkie dfbdf8d3bb Add a basic readme with link to the mixin docs.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:23:14 +00:00
Tom Wilkie 5fd712b210 copypasta.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Tom Wilkie 50861d586a Alert if more than 1% of alerts fail for a given integration.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Tom Wilkie 266ba185fe Remove PromScrapeFailed alert.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Tom Wilkie e8a8ce5654 Basic Prometheus dashboard.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Tom Wilkie ee1427faad Prometheus monitoring mixin for Prometheus itself.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-11-16 17:17:47 +00:00
Timo Beckers b12ed54f95
documentation - add marathon-sd example configuration
Signed-off-by: Timo Beckers <timo@incline.eu>
2018-10-25 18:02:59 +02:00
Brian Pandola 3241c527d0 Fix typo (#4760)
Signed-off-by: Brian Pandola <bpandola@hsdp.io>
2018-10-18 21:19:21 +01:00
Simon Pasquier 07152ecc48
Merge pull request #4575 from Nexucis/bugfix/fix-unregistered-source
[ServiceDiscovery] Unregister source when the target is empty
2018-09-27 09:12:01 +02:00
Augustin Husson f60620ec0b format comment
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2018-09-26 10:48:35 +02:00
Tom Wilkie d3a1ff1abf
Reduce memory usage of remote read by reducing pointer usage. (#4655)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-09-25 19:14:00 +01:00
Augustin Husson 9e6dc6f96c fix targetGroup that disappear totally
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2018-09-25 19:05:02 +02:00
Augustin Husson 3c0b130e5e apply review
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2018-09-18 11:08:38 +02:00
beorn7 4fb59d1e61 Remove use of deprecated prometheus.Handler
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-09-17 13:05:43 +02:00
Augustin Husson e03869de76 add unit test and isolate the method that generate the target
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2018-09-16 23:50:10 +02:00
Augustin Husson 97950a3fae remove group if the target is empty at adapter level
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2018-09-16 14:38:21 +02:00
Julius Volz 8fbe1b5133
Handle a bunch of unchecked errors (#4461)
There are many more (mostly finalizers like Close/Stop/etc.), but most of
the others seemed like one couldn't do much about them anyway.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-08-17 17:24:35 +02:00
Harsh Agarwal 6a464ae174 expose log.level for promlog for remote_storage_adapter (#4195)
* expose log.level for promlog for remote_storage_adapter

Signed-off-by: sipian <cs15btech11019@iith.ac.in>

* replace flag description

Signed-off-by: Harsh Agarwal <cs15btech11019@iith.ac.in>
2018-07-22 16:11:38 +05:30
Julius Volz d8153ac5d5
Update internal architecture diagram (#4398)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-07-18 22:10:23 +02:00
Julius Volz a215aed9b6
Document internal Prometheus server architecture (#4295)
* Document internal Prometheus server architecture

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Review fixups

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-07-18 10:06:41 +02:00
Peter Gallerani a9d5034add Fix missing 'msg' in remote storage adapter main.go .Log info message (#4377)
Signed-off-by: Peter Gallerani <peter.gallerani@gmail.com>
2018-07-12 20:54:21 +02:00
Callum Styan d0ee4da932 fix minor issues in custom SD example (#4278)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2018-06-18 16:08:02 +01:00
Callum Styan 03578d5df8 add example usage of SD adapter for converting unsupported SD type to filesd (#3720)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2018-05-30 13:14:34 +01:00
Bartek Plotka 03a9e7f72e example: Commented out annotation examples as they are meant only for example not as an idiomatic way of relabelling.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2018-05-02 13:42:23 +01:00
Paul Gier 85a3c974b7 minor yaml indentation consistency fix in example configs (#3946) 2018-03-11 23:06:13 +00:00
ferhat elmas ffa673f7d8 General simplifications (#3887)
Another try as in #1516
2018-02-26 07:58:10 +00:00
Goutham Veeramachaneni 3de10e3b44
Add CleanTombstones API endpoint
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-11-30 19:51:44 +05:30
root 80e5867a87 Fixed RBAC Example, added ingress privileges; @brian-brazil 2017-11-21 11:04:07 +01:00
Matthew Pound e6dcc72f9a Fix instructions on updating prometheus.yml for Remote Write Adapter Example (#3422) 2017-11-07 20:02:43 +00:00
James Turnbull 4db6592d01 Removing external_labels from example conf file (#3409)
It's unclear why this is in the example configuration file. Probably best to keep that super simple, c.f. https://github.com/prometheus/docs/pull/895#discussion_r148924390
2017-11-06 16:11:04 +00:00
David e3b926c03b Fix typo in AM config field static_configs (#3415)
* typo in prometheus.yml field causes prometheus to throw an error

Fixes #3414
2017-11-06 09:46:09 +00:00
Krasi Georgiev 5d8f93a22a now using only github.com/gogo/protobuf
bumped all grpc-gateway packages to v1.2.2
updated and run  the denproto.sh script
2017-11-02 11:31:57 +00:00
Julien Pivotto 3382f39046 Explicitely add alertmanager to example config (#3383)
As alertmanager needs to be configured in the config file in Prometheus 2, I think it is useful to have it in the example config.

Also renamed the rules in the example config so they are explicitely yml files.
2017-10-31 22:02:08 +00:00
Julius Volz 099df0c5f0 Migrate "golang.org/x/net/context" -> "context" (#3333)
In some places, where ctxhttp or gRPC are concerned, we still need to use the
old contexts.
2017-10-24 21:21:42 -07:00
Fabian Reinartz abf7c975c9 Merge branch 'master' into dev-2.0 2017-10-07 13:37:21 +02:00
Jack Neely 128b31d058 Log failure to send NaN values to remote store as Debug (#3235)
This was a warning and can be a frequent occurrence.  Let's not fill up
logs unless we are asked to.
2017-10-06 11:22:55 +01:00
Goutham Veeramachaneni 3f0267c548 Merge branch 'dev-2.0' into go-kit/log
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-15 23:15:27 +05:30
Goutham Veeramachaneni f5aed810f9 logging: Port to common/promlog
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-15 12:40:50 +05:30
Fabian Reinartz e746282772 Merge branch 'master' into dev-2.0 2017-09-11 10:55:19 +02:00
Fabian Reinartz d21f149745 *: migrate to go-kit/log 2017-09-08 22:01:51 +05:30
Fabian Reinartz 87918f3097 Merge branch 'master' into dev-2.0 2017-09-04 14:09:21 +02:00
Johannes 'fish' Ziemke 70f3d1e9f9 k8s: Support discovery of ingresses (#3111)
* k8s: Support discovery of ingresses

* Move additional labels below allocation

This makes it more obvious why the additional elements are allocated.
Also fix allocation for node where we only set a single label.

* k8s: Remove port from ingress discovery

* k8s: Add comment to ingress discovery example
2017-09-04 13:10:44 +02:00
Julius Volz aa5cdcb11e Remove extra space in log output 2017-08-29 15:24:00 +02:00
gdmello 35c952e344 Added logging for remote storage adapter (#3106)
* Added logging for remote storage adapter on startup and on any error condition during /read or /write.

* CR feedback.
2017-08-29 15:22:56 +02:00
Fabian Reinartz 25f3e1c424 Merge branch 'master' into mergemaster 2017-08-10 17:04:25 +02:00
Felicity f30b10223a documentation: update Kubernetes example for 1.7 (#2918)
Kubernetes 1.7+ no longer exposes cAdvisor metrics on the Kubelet
metrics endpoint.  Update the example configuration to scrape cAdvisor
in addition to Kubelet.  The provided configuration works for 1.7.3+
and commented notes are given for 1.7.2 and earlier versions.

Also remove the comment about node (Kubelet) CA not matching the master
CA.  Since the example no longer connects directly to the nodes, it
doesn't matter what CA they're using.

References:

- https://github.com/kubernetes/kubernetes/issues/48483
- https://github.com/kubernetes/kubernetes/pull/49079
2017-07-21 14:10:02 +02:00
Tom Wilkie cf105f9d57 Update example remote adapters for change in proto location. 2017-07-19 16:39:02 +01:00
Fabian Reinartz 32226e30f5 Guard reload and quit endpoints by flag 2017-07-11 14:25:07 +02:00
Fabian Reinartz ccf9e62972 *: add admin grpc API 2017-07-10 09:14:14 +02:00
Julius Volz e0f046396a Fix InfluxDB retention policy usage in read adapter (#2781) 2017-05-29 16:24:24 +02:00
Tom Wilkie 3141a6b36b Compress remote storage requests and responses with unframed/raw snappy. (#2696)
* Compress remote storage requests and responses with unframed/raw snappy, for compatibility with other languages.

* Remove backwards compatibility code from remote_storage_adapter, update example_write_adapter

* Add /documentation/examples/remote_storage/example_write_adapter/example_writer_adapter to .gitignore
2017-05-10 16:42:59 +02:00
Jorrit Salverda 14d0604aba Kubernetes config scrape node via api proxy (#2641)
* scrape kubelet metrics via api node proxy

* add manifests to setup serviceaccount, clusterrole and clusterrolebinding to work with rbac

* removed .cluster.local and added newline to address comments
2017-05-09 13:57:49 +02:00
Svend Sorensen 94a3e863e4 Document what ports are scraped by default in k8s example
The Kubernetes pod SD creates a target for each declared port, as documented:

https://prometheus.io/docs/operating/configuration/#pod

> The pod role discovers all pods and exposes their containers as targets. For
> each declared port of a container, a single target is generated. If a
> container has no specified ports, a port-free target per container is created
> for manually adding a port via relabeling.

This results in the default port being the declared port, or no port if none are
declared.
2017-05-01 15:58:48 -07:00
Brian Brazil 0e0fc5a7f4 Correct example name to adapter. (#2590) 2017-04-10 17:24:53 +01:00
Brian Brazil c813c824d4 Separate out remote read responses.
Fixes #2574
2017-04-06 15:49:47 +01:00
Julius Volz 3581057ea4 Update remote storage bridge README.md 2017-04-03 01:42:49 +02:00