Commit graph

23 commits

Author SHA1 Message Date
gotjosh 37b408c6cd
Feature: Allow configuration of a rule evaluation delay (#14061)
* [PATCH] Allow having evaluation delay for rule groups

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* [PATCH] Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* [PATCH] Move the option to ManagerOptions

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* [PATCH] Include evaluation_delay in the group config

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix comments

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Add a server configuration option.

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Appease the linter #1

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Add the new server flag documentation

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Improve documentation of the new flag and configuration

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Use named parameters for clarity on the `Rule` interface

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Add `initial` to the flag help

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Change the CHANGELOG area from `ruler` to `rules`

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Rename evaluation_delay to `rule_query_offset`/`query_offset` and make it a global configuration option.

Signed-off-by: gotjosh <josue.abreu@gmail.com>

E Your branch is up to date with 'origin/gotjosh/evaluation-delay'.

* more docs

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Improve wording on CHANGELOG

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Add `RuleQueryOffset` to the default config in tests in case it changes

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Update docs/configuration/recording_rules.md

Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Rename `RuleQueryOffset` to `QueryOffset` when in the group context.

Signed-off-by: gotjosh <josue.abreu@gmail.com>

* Improve docstring and documentation on the `rule_query_offset`

Signed-off-by: gotjosh <josue.abreu@gmail.com>

---------

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: gotjosh <josue.abreu@gmail.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
2024-05-30 11:49:50 +01:00
Jennifer Villa ea7bec4cc7
Update recording_rules.md
updated language to be a bit more clear

Signed-off-by: Jennifer Villa <jvilla2013@gmail.com>
2023-10-02 21:44:54 -04:00
Jennifer Villa 65ffa0c511
Update recording_rules.md
adding Josh's suggestion

Signed-off-by: Jennifer Villa <jvilla2013@gmail.com>
2023-10-02 21:37:29 -04:00
Jennifer Villa 601e5b9028
Update recording_rules.md
specified the evaluation interval defines when the next evaluation should start.

Signed-off-by: Jennifer Villa <jvilla2013@gmail.com>
2023-09-30 10:26:29 -04:00
Jennifer Villa 5027863c40
Clarify what happens when a rule group takes too long to execute
Namely, call out that all subsequent evaluations will be skipped until the initial evaluation completes.

Signed-off-by: Jennifer Villa <jvilla2013@gmail.com>
2023-09-30 10:23:54 -04:00
Julien Pivotto ce55e5074d Add 'keep_firing_for' field to alerting rules
This commit adds a new 'keep_firing_for' field to Prometheus alerting
rules. The 'resolve_delay' field specifies the minimum amount of time
that an alert should remain firing, even if the expression does not
return any results.

This feature was discussed at a previous dev summit, and it was
determined that a feature like this would be useful in order to allow
the expression time to stabilize and prevent confusing resolved messages
from being propagated through Alertmanager.

This approach is simpler than having two PromQL queries, as was
sometimes discussed, and it should be easy to implement.

This commit does not include tests for the 'resolve_delay' field.  This
is intentional, as the purpose of this commit is to gather comments on
the proposed design of the 'resolve_delay' field before implementing
tests. Once the design of the 'resolve_delay' field has been finalized,
a follow-up commit will be submitted with tests."

See https://github.com/prometheus/prometheus/issues/11570

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-01-13 12:11:39 +01:00
Ganesh Vernekar b4e15899d1
docs: Update recording rule docs about native histograms
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-01-12 20:43:02 +05:30
Sam Jewell f88a0a7d83
Update example rules file to be valid with the default scrape config (#11692)
* Update docs example rules for default config

The prometheus download includes a default config to scrape itself.
This self-scraping prometheus doesn't include any metric named as
`http_inprogress_requests`, but does include one named
`prometheus_http_requests_total`.
Updating this example rule in the docs to one which can be used
out-of-the-box with the default download would be a nice improvement.

Signed-off-by: Sam Jewell <sam.jewell@grafana.com>

* Update syntax as per @LeviHarrison's review

Co-authored-by: Levi Harrison <levisamuelharrison@gmail.com>
Signed-off-by: Sam Jewell <2903904+samjewell@users.noreply.github.com>

Signed-off-by: Sam Jewell <sam.jewell@grafana.com>
Signed-off-by: Sam Jewell <2903904+samjewell@users.noreply.github.com>
Co-authored-by: Levi Harrison <levisamuelharrison@gmail.com>
2023-01-09 19:36:07 -05:00
John Carlo Roberto 924ba90c3f
Add link to best practices in "Defining Recording Rules" page (#11696)
* docs: Add link to best practices in "Defining Recording Rules" page

Signed-off-by: John Carlo Roberto <10111643+Irizwaririz@users.noreply.github.com>

* docs: Improve wording

Signed-off-by: John Carlo Roberto <10111643+Irizwaririz@users.noreply.github.com>

Signed-off-by: John Carlo Roberto <10111643+Irizwaririz@users.noreply.github.com>
2022-12-12 16:08:45 +01:00
Björn Rabenstein 1c798ec930
doc: Add notes about feature not yet supported for native histograms (#11453)
Namely federation and recording rules.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-10-14 16:35:27 +05:30
Levi Harrison d81bbe154d
Rule alerts/series limit updates (#9541)
* Add docs and do not limit inactive alerts.

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-10-21 23:14:17 +02:00
Levi Harrison dc2f1993d8
Limit number of alerts or series produced by a rule (#9260)
* Add limit to rules

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-15 09:48:26 +02:00
beorn7 ae900a117f Recommend to get promtool from a binary distribution.
Rather than compile it yourself, which doesn't work as shown anymore
because of Go Modules.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-02-15 21:59:32 +01:00
Thibault Jamet f055690b34
Clarify: all rules in a group are concomitant (#8248)
Improve the documentation to clarify the differences beetween rules in a
group and outside a group.

Signed-off-by: Thibault Jamet <tjamet@users.noreply.github.com>
2020-12-03 10:32:10 +00:00
Peter Wu 1797192f02
Fix the alerting rules name description (#7083) (#8197)
commit 9875afc491 changed the type from
metric names to label values, we might as well adjust the description.
The alternative is to revert that commit and restrict names of alerting
rules again even if that was not really enforced.

Signed-off-by: Peter Wu <pwu@cloudflare.com>
2020-11-18 19:29:01 +00:00
Julien Pivotto 9875afc491
Fix the type of the alert name (#7523)
The alert name should be a valid label value, not a metric name.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-06 23:00:16 +02:00
Harold Dost 18d45e564b
Documentation: Update example expressions to follow convention. (#7195)
Based out of conversation on #7193

Signed-off-by: Harold Dost <h.dost@criteo.com>
2020-05-02 12:52:24 +01:00
Andrew Hayworth a336908678
Adds link to valid metric names (#6774)
One of our users today asked us if dashes were allowed in recording rule names.
We asserted that they were not, but also that we could not remember for certain.
After determining empirically that they are _not_ allowed, I realized that the
documentation could be slightly clearer about valid rule names.

This PR simply adds a note to the documentation re-iterating that the rules must
be valid metric names - and more importantly, adds a link to where a user can
read what those *are*, in case they were not aware (or did not know where to find it).

Signed-off-by: Andrew Hayworth <ahayworth@gmail.com>
2020-02-07 07:32:15 +00:00
vthriller b4bd91958a [minor] docs: recording_rules: fix missing key 2017-12-14 12:20:28 +00:00
phyber 013dc30dee Fix markdown in recording rules. (#3432)
Resolves an issue where rendered markdown was incorrect.
2017-12-14 12:20:28 +00:00
Goutham Veeramachaneni 646e33242e docs: Fix minor issues with the docs. (#3389)
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-11-01 15:35:50 +00:00
Brian Brazil b6494960d1
docs: Document new recording rule format (#3378) 2017-11-01 12:58:32 +00:00
Tobias Schmidt f432b8176d Consolidate configuration and rules docs in docs/configuration/ 2017-10-27 09:54:02 +02:00
Renamed from docs/querying/rules.md (Browse further)