* Add draining of queued notifications to `notifier.Manager`
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Update docs
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Address PR feedback
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Add more logging
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Address offline feedback: remove timeout
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Ensure stopping takes priority over further processing, make tests more robust
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Make channel unbuffered
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Update docs
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Fix race in test
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Remove unnecessary context
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* Make Stop safe to call multiple times
Signed-off-by: Charles Korn <charles.korn@grafana.com>
---------
Signed-off-by: Charles Korn <charles.korn@grafana.com>
* discovery: aws: expose Primary IPv6 addresses as label
Add __meta_ec2_primary_ipv6_addresses label. This label contains the
Primary IPv6 address for every ENI attached to the EC2 instance. It is
ordered by the DeviceIndex and the missing elements (interface without
Primary IPv6 address) are kept in the list.
---------
Signed-off-by: Arpad Kunszt <akunszt@hiya.com>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
* clarify backup requirements for storage
After reading this (again) recently, I was under the impression that our backup strategy ("just throw Bacula at it") was just not good enough and that our backups were inconsistent. I filed [an issue internally][41627] about this because of that concern.
But reading a conversation with @SuperQ on IRC, I came under the impression that only the WAL files would be lost. This is an attempt at documenting this more clearly.
[41627]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/41627
---------
Signed-off-by: anarcat <anarcat@users.noreply.github.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
Add the ability to adjust the `GOGC` variable from the Prometheus
configuration file.
* Create a new top-level `runtime` section in the config.
* Adjust from the Go default of 100 to 50 to reduce wasted memory.
* Use the `GOGC` env value if no configuraiton is used.
Signed-off-by: SuperQ <superq@gmail.com>
* [PATCH] Allow having evaluation delay for rule groups
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* [PATCH] Fix lint
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* [PATCH] Move the option to ManagerOptions
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* [PATCH] Include evaluation_delay in the group config
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* Fix comments
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Add a server configuration option.
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Appease the linter #1
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Add the new server flag documentation
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Improve documentation of the new flag and configuration
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Use named parameters for clarity on the `Rule` interface
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Add `initial` to the flag help
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Change the CHANGELOG area from `ruler` to `rules`
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Rename evaluation_delay to `rule_query_offset`/`query_offset` and make it a global configuration option.
Signed-off-by: gotjosh <josue.abreu@gmail.com>
E Your branch is up to date with 'origin/gotjosh/evaluation-delay'.
* more docs
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Improve wording on CHANGELOG
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Add `RuleQueryOffset` to the default config in tests in case it changes
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Update docs/configuration/recording_rules.md
Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Rename `RuleQueryOffset` to `QueryOffset` when in the group context.
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Improve docstring and documentation on the `rule_query_offset`
Signed-off-by: gotjosh <josue.abreu@gmail.com>
---------
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: gotjosh <josue.abreu@gmail.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
This is about native histograms (not yet supported) and staleness
markers (for which OpenMetrics support isn't even planned).
Signed-off-by: beorn7 <beorn@grafana.com>
docs: Remove outdated information about remote-read API
---------
Signed-off-by: kushagra Shukla <kushalshukla110@gmail.com>
Signed-off-by: Kushal shukla <85934954+kushalShukla-web@users.noreply.github.com>
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
Co-authored-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* fix(scaleway-sd): use public IPs if no private IP present
* tests(scaleway-sd): add instance with routed public ip and no private ip
---------
Signed-off-by: Heyoxe <32708033+Heyoxe@users.noreply.github.com>
This keeps the old "react-app" directory in its existing location (to make
it easier to merge changes from the main branch), but separates it from the
npm workspaces setup. Thus it now needs to be npm-installed/built/linted
separately. This is a bit hacky, but should only be needed temporarily,
until the old UI can be removed.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
This commit adds 2 new metadata labels for the endpointslice role:
* `__meta_kubernetes_endpointslice_endpoint_node_name`
* `__meta_kubernetes_endpointslice_endpoint_zone`
The latter is only present when the `discovery.k8s.io/v1` API group is
available.
I also updated the configuration doc and added an entry for the
`__meta_kubernetes_endpointslice_endpoint_hostname` label which was
missing.
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
use it in loadDataAsQueryable to make sure the RO Head doesn't truncate or cut new chunks in data/chunks_head/.
add a -sandbox-dir-root flag to "promtool tsdb dump/dump-openmetrics" to control the root of that sandbox dirrectory.
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
* Add support for discovering Linode IPv6 ranges associated with linodes.
* Add optional but recommended region filtering (faster queries, more relevant information).
* Added missing fields in configuration.md, updated linode test cases.
* Convert to TableDrivenTests as per tjhop request.
Signed-off-by: David Andruczyk <dandrucz@akamai.com>
This is the time period covered by a batch of samples, when the
number of waiting samples is lower than max_samples_per_send.
It does not affect timeouts or retries.
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Federico Leva <federico.leva@relexsolutions.com>
Support limit parameter in queries to restrict output data to the specified size, on the following endpoints:
/api/v1/series
/api/v1/labels
/api/v1/label/:name:/values
Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>
Signed-off-by: Kartikay <kartikay_2101ce32@iitp.ac.in>
This closes the loop, as the output can be fed into "tsdb create-blocks-from openmetrics"
Native histograms are not supported.
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
Updated & added tests
Review feedback nits
Return empty map if not indeterminate
Use highWatermark to track inflight requests counter
Appease the linter
Clarify feature flag
Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
Enable automatic detection of memory limits and configure GOMEMLIMIT to
match.
* Also includes a flag to allow controlling the reserved ratio.
Signed-off-by: SuperQ <superq@gmail.com>
Native Histograms: support native_histogram_min_bucket_factor in scrape_config
---------
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
Signed-off-by: Björn Rabenstein <github@rabenste.in>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
Conditions are ANDed inside the same matcher but matchers are ORed
Including unit tests for "promtool tsdb dump".
Refactor some matchers scraping utils.
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
Add `query analyze` command to promtool
This command analyzes the buckets of classic and native histograms,
based on data queried from the Prometheus query API, i.e. it
doesn't require direct access to the TSDB files.
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
---------
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
* Drop old inmemory samples
Co-authored-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Avoid copying timeseries when the feature is disabled
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Run gofmt
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Clarify docs
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Add more logging info
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Remove loggers
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* optimize function and add tests
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Simplify filter
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* rename var
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Update help info from metrics
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* use metrics to keep track of drop elements during buildWriteRequest
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* rename var in tests
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* pass time.Now as parameter
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Change buildwriterequest during retries
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Revert "Remove loggers"
This reverts commit 54f91dfcae20488944162335ab4ad8be459df1ab.
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* use log level debug for loggers
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Fix linter
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
* Remove noisy debug-level logs; add 'reason' label to drop metrics
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
* Remove accidentally committed files
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
* Propagate logger to buildWriteRequest to log dropped data
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
* Fix docs comment
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
* Make drop reason more specific
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
* Remove unnecessary pass of logger
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
* Use snake_case for reason label
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
* Fix dropped samples metric
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
---------
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
Signed-off-by: Paschalis Tsilias <tpaschalis@users.noreply.github.com>
Co-authored-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
Co-authored-by: Paschalis Tsilias <tpaschalis@users.noreply.github.com>
promql: Improve histogram_quantile calculation for classic buckets
Tiny differences between classic buckets are most likely caused by floating point precision issues. With this commit, relative changes below a certain threshold are ignored. This makes the result of histogram_quantile more meaningful, and also avoids triggering the _input to histogram_quantile needed to be fixed for monotonicity_ annotations in unactionable cases.
This commit also adds explanation of the new adjustment and of the monotonicity annotation to the documentation of `histogram_quantile`.
---------
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Currently Prometheus will always request gzip compression from the target when sending scrape requests.
HTTP compression does reduce the amount of bytes sent over the wire and so is often desirable.
The downside of compression is that it requires extra resources - cpu & memory.
This also affects the resource usage on the target since it has to compress the response
before sending it to Prometheus.
This change adds a new option to the scrape job configuration block: enable_compression.
The default is true so it remains the same as current Prometheus behaviour.
Setting this option to false allows users to disable compression between Prometheus
and the scraped target, which will require more bandwidth but it lowers the resource
usage of both Prometheus and the target.
Fixes#12319.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
This PR adds an Experimental flag to the functions.
This can be used by https://github.com/prometheus/prometheus/pull/13059
but also xrate and other future functions.
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
* Document le and quantile label transition due to native histograms
Fixes: #12984
For full explanation see the related issue. The le and quantile labels
are formatted as float with trailing .0 for whole number values when
native histograms is enabled, e.g. 10.0. This changes the resulting series
in Prometheus if previously we scraped the whole number itself, e.g. 10
over the text format.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
---------
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
* Added ability to specify scrape protocols to accept during HTTP content type negotiation.
This is done via new option in GlobalConfig and ScrapeConfig: "scrape_protocol"
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Fixed readability and log message.
Signed-off-by: bwplotka <bwplotka@gmail.com>
---------
Signed-off-by: bwplotka <bwplotka@gmail.com>
Namely, call out that all subsequent evaluations will be skipped until the initial evaluation completes.
Signed-off-by: Jennifer Villa <jvilla2013@gmail.com>
* support specifying series matchers to analyze tsdb
Signed-off-by: Ben Ye <benye@amazon.com>
* fix cli docs
Signed-off-by: Ben Ye <benye@amazon.com>
---------
Signed-off-by: Ben Ye <benye@amazon.com>
Addresses: #12536
This commit adds support for configuring sigv4 to an
`alertmanager_config`. Based heavily on the sigv4 work in the remote
write client.
Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>
promql: Extend testing framework to support native histograms
This includes both the internal testing framework as well as the rules unit test feature of promtool.
This also adds a bunch of basic tests. Many of the code level tests can now be converted to tests within the framework, and more tests can be added easily.
---------
Signed-off-by: Harold Dost <h.dost@criteo.com>
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
Signed-off-by: Stephen Lang <stephen.lang@grafana.com>
Co-authored-by: Harold Dost <h.dost@criteo.com>
Co-authored-by: Stephen Lang <stephen.lang@grafana.com>
Co-authored-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
It's possible (quite common on Kubernetes) to have a service discovery
return thousands of targets then drop most of them in relabel rules.
The main place this data is used is to display in the web UI, where
you don't want thousands of lines of display.
The new limit is `keep_dropped_targets`, which defaults to 0
for backwards-compatibility.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
We still need a guide that we can link users to in https://github.com/prometheus/docs/tree/main/content/docs/guides
This guide should show sending metrics from application directly via
the OTel SDKs and also sending through the Collector.
Signed-off-by: Goutham <gouthamve@gmail.com>
* Add OTLP Ingestion endpoint
We copy files from the otel-collector-contrib. See the README in
`storage/remote/otlptranslator/README.md`.
This supersedes: https://github.com/prometheus/prometheus/pull/11965
Signed-off-by: gouthamve <gouthamve@gmail.com>
* Return a 200 OK
It is what the OTEL Golang SDK expect :(
https://github.com/open-telemetry/opentelemetry-go/issues/4363
Signed-off-by: Goutham <gouthamve@gmail.com>
---------
Signed-off-by: gouthamve <gouthamve@gmail.com>
Signed-off-by: Goutham <gouthamve@gmail.com>
Adds a new label to include the ID of the image that an instance is
using. This can be used for example to filter a job to only include
instances using a certain image as that image includes some exporter.
Sometimes the image information isn't available, such as when the image
is private and the user doesn't have the roles required to see it. In
those cases we just don't set the label, as the rest of the information
from the discovery provider can still be used.
Signed-off-by: Taavi Väänänen <hi@taavi.wtf>
Just adding a statement here explaining that the default is an
immediate move to "active" without a pending state.
Signed-off-by: Tim Martin <tim@timmartin.me>
Adds web config option `client_allowed_sans`. This enables Prometheus to
limit the Subject Alternate Name (SAN) allowed to connect.
Signed-off-by: SuperQ <superq@gmail.com>
