This affects the following alerts: - NodeFilesystemAlmostOutOfFiles - NodeFilesystemSpaceFillingUp - NodeFilesystemAlmostOutOfSpace - NodeFilesystemFilesFillingUp Each alert is actually two alerting rules, one for the warning level, and one for the critical level. However, the critical one can be set to emit as a warning by using $._config.nodeCriticalSeverity. When this is true, both alerting rules will generate alerts with an identical set of labels, which is an error when both alerts fire. We add an extra label to these alerting rules in order to differentiate between the two levels when $._config.nodeCriticalSeverity is set to `warning`. Signed-off-by: Justin Lei <lei.justin@gmail.com> |
||
---|---|---|
.. | ||
alerts | ||
dashboards | ||
lib | ||
rules | ||
.gitignore | ||
alerts.jsonnet | ||
config.libsonnet | ||
dashboards.jsonnet | ||
jsonnetfile.json | ||
Makefile | ||
mixin.libsonnet | ||
README.md | ||
rules.jsonnet |
Node Mixin
This is a work in progress. We aim for it to become a good role model for alerts and dashboards eventually, but it is not quite there yet.
The Node Mixin is a set of configurable, reusable, and extensible alerts and dashboards based on the metrics exported by the Node Exporter. The mixin creates recording and alerting rules for Prometheus and suitable dashboard descriptions for Grafana.
To use them, you need to have jsonnet
(v0.16+) and jb
installed. If you
have a working Go development environment, it's easiest to run the following:
go install github.com/google/go-jsonnet/cmd/jsonnet@latest
go install github.com/google/go-jsonnet/cmd/jsonnetfmt@latest
go install github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@latest
Next, install the dependencies by running the following command in this directory:
jb install
You can then build the Prometheus rules files node_alerts.yaml
and
node_rules.yaml
:
make node_alerts.yaml node_rules.yaml
You can also build a directory dashboard_out
with the JSON dashboard files
for Grafana:
make dashboards_out
Note that some of the generated dashboards require recording rules specified in
the previously generated node_rules.yaml
.
For more advanced uses of mixins, see https://github.com/monitoring-mixins/docs.