This very simple tool dumps all samples in a LevelDB metrics storage
together with their chunk information. This is very crude for now - I
just whipped it up because I needed to debug something. We should add
printf-style formatting support for it in the future to indicate what
information to output for each sample in the database. We could also
support inspecting the index databases.
The previous implementation spawned N goroutines to group samples
together and would not start work until the semaphore unblocked.
While this didn't leak, it polluted the scheduling space. Thusly,
the routine only starts after a semaphore has been acquired.
The one-off keys have been replaced with ``model.LabelPair``, which is
indexable. The performance impact is negligible, but it represents
a cognitive simplification.
The reality is that if we ever try to encode a Protocol Buffer and it
fails, it's likely that such an error is ultimately not a runtime error
and should be fixed forthwith. Thusly, we should rename
``Encoder.Encode`` to ``Encoder.MustEncode`` and drop the error return
value.
Some users of GetMetricForFingerprint() end up modifying the returned metric
labelset. Since the memory storage's implementation of
GetMetricForFingerprint() returned a pointer to the metric (and maps are
reference types anyways), the external mutation propagated back into the memory
storage.
The fix is to make a copy of the metric before returning it.
- only the data extracted in the last loop iteration of ExtractSamples() was
emitted as output
- if e.g. op interval < sample interval, there were situations where the same
sample was added multiple times to the output
This commit updates the documentation, Makefiles, formatting, and
code semantics to support the 1.1. runtime, which includes ...
1. ``make advice``,
2. ``make format``, and
3. ``go fix`` on various targets.
This commit simplifies the way that compactions across a database's
keyspace occur due to reading the LevelDB internals. Secondarily it
introduces the database size estimation mechanisms.
Include database health and help interfaces.
Add database statistics; remove status goroutines.
This commit kills the use of Go routines to expose status throughout
the web components of Prometheus. It also dumps raw LevelDB status
on a separate /databases endpoint.
This commit simplifies the way that compactions across a database's
keyspace occur due to reading the LevelDB internals. Secondarily it
introduces the database size estimation mechanisms.
This commit introduces the long-tail deletion mechanism, which will
automatically cull old sample values. It is an acceptable
hold-over until we get a resampling pipeline implemented.
Kill legacy OS X documentation, too.
This does two things:
1) Make TieredStorage.AppendSamples() write directly to memory instead of
buffering to a channel first. This is needed in cases where a rule might
immediately need the data generated by a previous rule.
2) Replace the single storage mutex by two new ones:
- memoryMutex - needs to be locked at any time that two concurrent
goroutines could be accessing (via read or write) the
TieredStorage memoryArena.
- memoryDeleteMutex - used to prevent any deletion of samples from
memoryArena as long as renderView is running and
assembling data from it.
The LevelDB disk storage does not need to be protected by a mutex when
rendering a view since renderView works off a LevelDB snapshot.
The rationale against adding memoryMutex directly to the memory storage: taking
a mutex does come with a small inherent time cost, and taking it is only
required in few places. In fact, no locking is required for the memory storage
instance which is part of a view (and not the TieredStorage).
This commit extracts the model.Values truncation behavior into the actual
tiered storage, which uses it and behaves in a peculiar way—notably the
retention of previous elements if the chunk were to ever go empty. This is
done to enable interpolation between sparse sample values in the evaluation
cycle. Nothing necessarily new here—just an extraction.
Now, the model.Values TruncateBefore functionality would do what a user
would expect without any surprises, which is required for the
DeletionProcessor, which may decide to split a large chunk in two if it
determines that the chunk contains the cut-off time.
