Commit graph

369 commits

Author SHA1 Message Date
Levi Harrison faed8df31d
Enable reading consul token from file (#8926)
* Adopted common http client

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-12 00:06:59 +02:00
Julien Pivotto 9444698ae2
http_sd (#8839)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-06-11 18:04:45 +02:00
Frederic Hemberger 39a87fd9d2 consul_sd: Add namespace support for Consul Enterprise
Signed-off-by: Frederic Hemberger <mail@frederic-hemberger.de>
2021-06-09 16:35:02 +02:00
Levi Harrison 7bc11dcb06
React UI: Add Starting Screen (#8662)
* Added walreplay API endpoint

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added starting page to react-ui

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Documented the new endpoint

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed typos

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Co-authored-by: Julius Volz <julius.volz@gmail.com>

* Removed logo

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed isResponding to isUnexpected

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed width of progress bar

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed width of progress bar

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added DB stats object

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Updated starting page to work with new fields

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 2)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 3)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (and also implementing a method this time) (pt. 4)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (and also implementing a method this time) (pt. 5)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed const to let

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 6)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Remove SetStats method

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added comma

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed api

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed to triple equals

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed data response types

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Don't return pointer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed version

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed interface issue

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed pointer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed copying lock value error

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Co-authored-by: Julius Volz <julius.volz@gmail.com>
2021-06-05 15:29:32 +01:00
Julien Pivotto 6cbed8c569
Merge pull request #8878 from LeviHarrison/external-address-alerts
Access external address in alerts
2021-06-04 09:14:25 +02:00
Julien Pivotto 609ba54b8f
Mark body_size_limit as experimental. (#8886)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-06-02 16:32:08 +01:00
Julien Pivotto 20c6739adc
Merge pull request #8833 from hanjm/feature/add-scape-read-body-limit
Add body_size_limit to prevent bad targets response large body cause Prometheus server OOM (#8827)
2021-06-02 09:24:59 +02:00
TJ Hoplock dc22c65349
Add Linode Service Discovery (#8846)
* Add Linode Service Discovery

Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>
2021-06-01 20:32:36 +02:00
Levi Harrison 51e4e2a526
Updated docs
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-05-30 23:36:05 -04:00
hanjm 1df05bfd49 Add body_size_limit to prevent bad targets response large body cause Prometheus server OOM (#8827)
Signed-off-by: hanjm <hanjinming@outlook.com>
2021-05-29 07:05:42 +08:00
Ikko Ashimine 446e5cc160 Fix typo in storage.md
mulitple -> multiple

Signed-off-by: Ikko Ashimine <eltociear@gmail.com>
2021-05-20 20:32:03 +09:00
Sandro 0ffcddbee8
Fix indentation
Signed-off-by: Sandro Jäckel <sandro.jaeckel@gmail.com>
2021-05-16 05:27:05 +02:00
Callum Styan 8fd73b1d28
Add Exemplar Remote Write support (#8296)
* Write exemplars to the WAL and send them over remote write.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Update example for exemplars, print data in a more obvious format.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Add metrics for remote write of exemplars.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix incorrect slices passed to send in remote write.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* We need to unregister the new metrics.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address review comments

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Order of exemplar append vs write exemplar to WAL needs to change.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Several fixes to prevent sending uninitialized or incorrect samples with an exemplar. Fix dropping exemplar for missing series. Add tests for queue_manager sending exemplars

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Store both samples and exemplars in the same timeseries buffer to remove the alloc when building final request, keep sub-slices in separate buffers for re-use

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Condense sample/exemplar delivery tests to parameterized sub-tests

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Rename test methods for clarity now that they also handle exemplars

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Rename counter variable. Fix instances where metrics were not updated correctly

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Add exemplars to LoadWAL benchmark

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* last exemplars timestamp metric needs to convert value to seconds with
ms precision

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Process exemplar records in a separate go routine when loading the WAL.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address review comments related to clarifying comments and variable
names. Also refactor sample/exemplar to enqueue prompb types.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Regenerate types proto with comments, update protoc version again.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Put remote write of exemplars behind a feature flag.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address some of Ganesh's review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Move exemplar remote write feature flag to a config file field.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address Bartek's review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Don't allocate exemplar buffers in queue_manager if we're not going to
send exemplars over remote write.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Add ValidateExemplar function, validate exemplars when appending to head
and log them all to WAL before adding them to exemplar storage.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address more reivew comments from Ganesh.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Add exemplar total label length check.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address a few last review comments

Signed-off-by: Callum Styan <callumstyan@gmail.com>

Co-authored-by: Martin Disibio <mdisibio@gmail.com>
2021-05-06 13:53:52 -07:00
Damien Grisonnet b50f9c1c84
Add label scrape limits (#8777)
* scrape: add label limits per scrape

Add three new limits to the scrape configuration to provide some
mechanism to defend against unbound number of labels and excessive
label lengths. If any of these limits are broken by a sample from a
scrape, the whole scrape will fail. For all of these configuration
options, a zero value means no limit.

The `label_limit` configuration will provide a mechanism to bound the
number of labels per-scrape of a certain sample to a user defined limit.
This limit will be tested against the sample labels plus the discovery
labels, but it will exclude the __name__ from the count since it is a
mandatory Prometheus label to which applying constraints isn't
meaningful.

The `label_name_length_limit` and `label_value_length_limit` will
prevent having labels of excessive lengths. These limits also skip the
__name__ label for the same reasons as the `label_limit` option and will
also make the scrape fail if any sample has a label name/value length
that exceed the predefined limits.

Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>

* scrape: add metrics and alert to label limits

Add three gauge, one for each label limit to easily access the
limit set by a certain scrape target.
Also add a counter to count the number of targets that exceeded the
label limits and thus were dropped. This is useful for the
`PrometheusLabelLimitHit` alert that will notify the users that scraping
some targets failed because they had samples exceeding the label limits
defined in the scrape configuration.

Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>

* scrape: apply label limits to __name__ label

Apply limits to the __name__ label that was previously skipped and
truncate the label names and values in the error messages as they can be
very very long.

Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>

* scrape: remove label limits gauges and refactor

Remove `prometheus_target_scrape_pool_label_limit`,
`prometheus_target_scrape_pool_label_name_length_limit`, and
`prometheus_target_scrape_pool_label_value_length_limit` as they are not
really useful since we don't have the information on the labels in it.

Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
2021-05-06 09:56:21 +01:00
Levi Harrison fa184a5fc3
Add OAuth 2.0 Config (#8761)
* Introduced oauth2 config into the codebase

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-04-28 14:47:52 +02:00
n888 7c028d59c2
Add lightsail service discovery (#8693)
Signed-off-by: N888 <drifto@gmail.com>
2021-04-28 11:29:12 +02:00
nberkley f9e2dd0697
Add support for smaller block chunk segment allocations (#8478)
* Add support for --storage.tsdb.max-chunk-size to suport small chunks for space limited prometheus instances.

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update tsdb/compact.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update tsdb/db.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update cmd/prometheus/main.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Change naming scheme to

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Add a lower bound to --storage.tsdb.max-block-chunk-segment-size

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update storage.md to explain what a chunk segment is

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Apply suggestions from code review

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Force tests

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Fix code style

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-04-15 14:25:01 +05:30
Julien Pivotto ea6f6bba74
Enable parsing strings in humanize functions (#8682)
* Enable parsing strings in humanize functions

This is useful to humanize count_values or buckets labels.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-04-14 00:30:15 +02:00
Jess G 731545ad34
Add documentation for recording rule backfiller (#8674)
* add docs for rule backfiller

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-04-01 22:38:00 +02:00
Bartlomiej Plotka f93b95d16e
Merge release-2.26.0 branch to main. (#8678)
* Cut 2.26.0-rc.0 release. (#8640)

* Cut 2.26.0-rc.0

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Curate 2.26 changelog

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* bump codemirror-promql to v0.14.0 (#8653)

Signed-off-by: Augustin Husson <husson.augustin@gmail.com>

* fix: Wrap text in CodeMirror expression editor (#8665)

Fixes https://github.com/prometheus/prometheus/issues/8663

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Quick documentation of the exemplars feature on the disabled features (#8655)

page.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Cut 2.26.0 (#8676)

* Cut 2.26.0-rc.1

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Moved to full release.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Julien Pivotto <roidelapluie@inuits.eu>
Co-authored-by: Augustin Husson <husson.augustin@gmail.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
2021-03-31 18:16:37 +01:00
Julien Pivotto 0414534499
Merge pull request #6947 from prometheus/document-global-labels
Add new "globalUrl" field to /api/v1/targets docs
2021-03-30 22:41:47 +02:00
Julien Pivotto e635ca834b Add environment variable expansion in external label values
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-30 01:36:28 +02:00
Robert Jacob b253056163
Implement Docker discovery (#8629)
* Implement Docker discovery

Signed-off-by: Robert Jacob <xperimental@solidproject.de>
2021-03-29 22:30:23 +02:00
Julien Pivotto 5a6d244b00 Scaleway SD: Add the ability to read token from file
Prometheus adds the ability to read secrets from files. This add
this feature for the scaleway service discovery.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-25 00:52:33 +01:00
Andrew Starr-Bochicchio da8a8585f5 Add vpc label to docs.
Signed-off-by: Andrew Starr-Bochicchio <a.starr.b@gmail.com>
2021-03-24 17:05:16 -04:00
Julien Pivotto c0c36b1155
Improve promql-negative-offset docs (#8631)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-22 10:16:43 +01:00
Julien Pivotto 49016994ac Switch to alertmanager api v2
According to the 2.25 release notes, 2.26 should switch to alertmanager
api v2 by default.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-20 01:01:10 +01:00
Callum Styan 289ba11b79
Add circular in-memory exemplars storage (#6635)
* Add circular in-memory exemplars storage

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>

* Fix some comments, clean up exemplar metrics struct and exemplar tests.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix exemplar query api null vs empty array issue.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>
2021-03-16 15:17:45 +05:30
Julius Volz 1cba174182
Document lack of metadata deletion in deletion API (#8376)
* Document lack of metadata deletion in deletion API

See also https://github.com/prometheus/prometheus/issues/7932

I also fixed the other "NOTE" callouts to be formatted in such a way that our
docs system formats them as callout boxes.

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* WIP

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Improve wordings

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-03-15 23:09:29 +01:00
Julien Pivotto bab0fb81ef Rename to or ID
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-12 23:53:08 +01:00
Julien Pivotto d1c8550894 Scaleway SD: Add more labels to the instance role
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-12 01:04:15 +01:00
Rémy Léone f690b811c5
add support for scaleway service discovery (#8555)
Co-authored-by: Patrik <patrik@ptrk.io>
Co-authored-by: Julien Pivotto <roidelapluie@inuits.eu>

Signed-off-by: Rémy Léone <rleone@scaleway.com>
2021-03-10 15:10:17 +01:00
Julien Pivotto ad5ed416ba
Merge pull request #8487 from pschou/dev_neg_offset
allow negative offset
2021-03-08 22:18:45 +01:00
Julien Pivotto f2fc05fd5f
Merge pull request #8569 from roidelapluie/fix-default-resolution
Fix subqueries with default resolution in promql unit tests
2021-03-08 20:44:09 +01:00
Robert Fratto 5b78aa0649
Contribute grafana/agent sigv4 code (#8509)
* Contribute grafana/agent sigv4 code
* address review feedback
  - move validation logic for RemoteWrite into unmarshal
  - copy configuration fields from ec2 SD config
  - remove enabled field, use pointer for enabling sigv4
* Update config/config.go
* Don't provide credentials if secret key / access key left blank
* Add SigV4 headers to the list of unchangeable headers.
* sigv4: don't include all headers in signature
* only test for equality in the authorization header, not the signed date
* address review feedback
  1. s/httpClientConfigEnabled/httpClientConfigAuthEnabled
  2. bearer_token tuples to "authorization"
  3. Un-export NewSigV4RoundTripper
* add x-amz-content-sha256 to list of unchangeable headers
* Document sigv4 configuration
* add suggestion for using default AWS SDK credentials

Signed-off-by: Robert Fratto <robertfratto@gmail.com>
Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>
2021-03-08 12:20:09 -07:00
Julien Pivotto 5742a18590 Fix subqueries with default resolution in promql unit tests
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-03-07 09:20:04 +01:00
schou 65658091ed moved details of feature to disabled features.md
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-03-03 21:42:17 -05:00
schou 946dba6a7f small suggested changes
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-03-03 21:35:51 -05:00
pschou c15f15d6fd revert disabled_features.md to previous wording.
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-03-02 22:01:02 -05:00
Julien Pivotto 93c6139bc1 Support follow_redirect
This PR introduces support for follow_redirect, to enable users to
disable following HTTP redirects.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-02-26 22:50:56 +01:00
Harkishen-Singh 79ba53a6c4 Custom headers on remote-read and refactor implementation to roundtripper.
Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>
2021-02-26 17:20:29 +05:30
pschou f80b52be69
Merge branch 'main' into dev_neg_offset 2021-02-23 20:52:57 -05:00
schou 71712183ac minor wording updates
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:26:35 -05:00
schou 9621f6d549 wording for shift correctness
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:25:56 -05:00
schou 22cd48868a adding feature flag, promql-negative-offset
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:25:56 -05:00
schou 8aa9cc3978 add blob to basics documentation about negative shift
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:25:56 -05:00
Julien Pivotto 432d5ebc6c Rename default branch to main
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-02-22 20:28:02 +01:00
pschou aff3c702ab
promql: Add sgn, clamp and last_over_time functions (#8457)
* Add sgn, clamp and last_over_time functions

Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-20 16:34:52 +01:00
Julien Pivotto 8787f0aed7 Update common to support credentials type
Most of the backwards compat tests is done in common.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-02-18 23:28:22 +01:00
Julien Pivotto 2d172d0896
Merge pull request #8508 from prometheus/release-2.25
Merge back release 2.25
2021-02-17 16:26:34 +01:00