Commit graph

13 commits

Author SHA1 Message Date
Julius Volz e3415e953f Add notifications telemetry. 2013-07-31 12:40:56 +02:00
Matt Proud 7f0d816574 Schedule the background compactors to run.
This commit introduces three background compactors, which compact
sparse samples together.

1. Older than five minutes is grouped together into chunks of 50 every 30
   minutes.

2. Older than 60 minutes is grouped together into chunks of 250 every 50
   minutes.

3. Older than one day is grouped together into chunks of 5000 every 70
   minutes.
2013-05-07 17:14:04 +02:00
Matt T. Proud a48ab34dd0 Refresh Prometheus client API usage.
The client API has been updated per https://github.com/prometheus/client_golang/pull/9.
2013-04-28 19:40:30 +02:00
Matt T. Proud b3e34c6658 Implement batch database sample curator.
This commit introduces to Prometheus a batch database sample curator,
which corroborates the high watermarks for sample series against the
curation watermark table to see whether a curator of a given type
needs to be run.

The curator is an abstract executor, which runs various curation
strategies across the database.  It remarks the progress for each
type of curation processor that runs for a given sample series.

A curation procesor is responsible for effectuating the underlying
batch changes that are request.  In this commit, we introduce the
CompactionProcessor, which takes several bits of runtime metadata and
combine sparse sample entries in the database together to form larger
groups.  For instance, for a given series it would be possible to
have the curator effectuate the following grouping:

- Samples Older than Two Weeks: Grouped into Bunches of 10000
- Samples Older than One Week: Grouped into Bunches of 1000
- Samples Older than One Day: Grouped into Bunches of 100
- Samples Older than One Hour: Grouped into Bunches of 10

The benefits hereof of such a compaction are 1. a smaller search
space in the database keyspace, 2. better employment of compression
for repetious values, and 3. reduced seek times.
2013-04-27 17:38:18 +02:00
Matt T. Proud c53a72a894 Test data for the curator. 2013-03-27 18:13:43 +01:00
Matt T. Proud 1f7ed52b46 Start writing high watermarks. 2013-03-21 18:08:48 +01:00
Matt T. Proud 187cd4cdbc Extract indexing of Fingerprint to Metrics. 2013-03-21 18:08:48 +01:00
Matt T. Proud 532589f728 Extract Label Pair to Fingerprint indexing. 2013-03-21 18:08:48 +01:00
Matt T. Proud 84acfed061 Extract finding unindexed metrics. 2013-03-21 18:08:48 +01:00
Matt T. Proud 67300af137 Extract indexing to separate routine. 2013-03-21 18:08:47 +01:00
Matt T. Proud 8cc5cdde0b checkpoint. 2013-03-21 18:08:46 +01:00
Matt T. Proud f39b9c3c8e Checkpoint. 2013-03-21 18:06:51 +01:00
Matt T. Proud 41068c2e84 Checkpoint. 2013-03-21 18:06:51 +01:00
Renamed from storage/metric/leveldb/instrumentation.go (Browse further)