Commit graph

207 commits

Author SHA1 Message Date
juliusv 95400cb785 Merge pull request #290 from prometheus/fix/go-vet
Minor "go tool vet" cleanups
2013-06-07 06:52:48 -07:00
Julius Volz 558281890b Minor "go tool vet" cleanups 2013-06-07 15:34:41 +02:00
juliusv 615972dd01 Merge pull request #288 from prometheus/fix/curator/fallthrough-compaction-ordering
Fix fallthrough compaction value ordering.
2013-06-07 05:46:15 -07:00
Matt T. Proud 86f63b078b Fix fallthrough compaction value ordering.
We discovered a regression whereby data chunks could be appended out
of order if the fallthrough case was hit.
2013-06-07 14:41:00 +02:00
Julius Volz 7b9ee95030 Minor LevelDB watermark handling cleanups. 2013-06-06 23:56:31 +02:00
Julius Volz 84741b227d Use LRU cache to avoid querying stale series. 2013-06-06 23:56:19 +02:00
Julius Volz f98853d7b7 Fix type error in watermark list handling. 2013-06-06 23:56:14 +02:00
Matt T. Proud ef1d5fd8a2 Introduce semaphores for tiered storage.
This commit wraps the tiered storage access componnets in semaphores,
since we can handle several concurrent memory reads.
2013-06-06 18:16:18 +02:00
Matt T. Proud 819045541e Code Review: Make double-drain a panic. 2013-06-06 12:40:06 +02:00
Matt T. Proud e217a9fb41 Race Work: Make memory arena locks more coarse.
We can optimize these as needed later.
2013-06-06 12:08:20 +02:00
Matt T. Proud beaaf386e7 Add storage state guards and transition callbacks.
To ensure that we access tiered storage in the proper way, we have
guards now.
2013-06-06 11:52:09 +02:00
Matt T. Proud abb5353ade Merge pull request #283 from prometheus/feature/storage/consult-watermark
Include LRU cache for fingerprint watermarks.
2013-06-06 02:33:45 -07:00
Matt T. Proud 2c3df44af6 Ensure database access waits until it is started.
This commit introduces a channel message to ensure serving
state has been reached with the storage stack before anything attempts
to use it.
2013-06-06 10:42:21 +02:00
Matt T. Proud cbe2f3a7b1 Include LRU cache for fingerprint watermarks. 2013-06-06 10:13:18 +02:00
Julius Volz 51689d965d Add debug timers to instant and range queries.
This adds timers around several query-relevant code blocks. For now, the
query timer stats are only logged for queries initiated through the UI.
In other cases (rule evaluations), the stats are simply thrown away.

My hope is that this helps us understand where queries spend time,
especially in cases where they sometimes hang for unusual amounts of
time.
2013-06-05 18:32:54 +02:00
Matt T. Proud 8339a189cb Code Review: Fix seriesPresent scope.
The seriesPresent scope should be constrained to the scope of a
scanJob, since this is keyed to given series.
2013-06-04 13:16:59 +02:00
Matt T. Proud fe41ce0b19 Conditionalize disk initializations.
This commit conditionalizes the creation of the diskFrontier and
seriesFrontier along with the iterator such that they are provisioned
once something is actually required from disk.
2013-06-04 12:53:57 +02:00
Julius Volz a8468a2e5e Fix reversed disk flush cutoff behavior. 2013-05-28 16:14:30 +02:00
Julius Volz eb1f956909 Revert "Revert "Ensure that all extracted samples are added to view.""
This reverts commit 4b30fb86b4.
2013-05-28 14:36:03 +02:00
Matt T. Proud 4b30fb86b4 Revert "Ensure that all extracted samples are added to view."
This reverts commit 008314b5a8. By
running an automated git bisection described in
https://gist.github.com/matttproud-soundcloud/22a371a8d2cba382ea64
this commit was found.
2013-05-23 13:36:22 +02:00
Julius Volz 750f862d9a Use GetBoundaryValues() for non-counter deltas. 2013-05-22 19:13:47 +02:00
Julius Volz f2b48b8c4a Make getValuesAtIntervalOp consume all chunk data in one pass.
This is mainly a small performance improvement, since we skip past the last
extracted time immediately if it was also the last sample in the chunk, instead
of trying to extract non-existent values before the chunk end again and again
and only gradually approaching the end of the chunk.
2013-05-22 18:14:45 +02:00
Julius Volz 83d60bed89 extractValuesAroundTime() code simplification. 2013-05-22 18:14:45 +02:00
Julius Volz 008314b5a8 Ensure that all extracted samples are added to view.
The current behavior only adds those samples to the view that are extracted by
the last pass of the last processed op and throws other ones away. This is a
bug. We need to append all samples that are extracted by each op pass.

This also makes view.appendSamples() take an array of samples.
2013-05-22 18:14:37 +02:00
Matt T. Proud b586801830 Code Review: Fix to-disk queue infinite growth.
We discovered a bug while manually testing this branch on a live
instance, whereby the to-disk queue was never actually dumped to
disk.
2013-05-22 17:59:53 +02:00
Matt T. Proud 285a8b701b Code Review: Extend lock. 2013-05-22 17:59:53 +02:00
Matt T. Proud 2526ab8c81 Code Review: Extend lock scope for appending. 2013-05-22 17:59:53 +02:00
Matt T. Proud f994482d15 Code Review: Avenues for future improvemnet noted. 2013-05-22 17:59:53 +02:00
Matt T. Proud 298a90c143 Code Review: Initial arena size name. 2013-05-22 17:59:53 +02:00
Matt T. Proud c07abf8521 Initial move away from skiplist. 2013-05-22 17:59:53 +02:00
Matt T. Proud 74a66fd938 Spawn grouping of fingerprints with free semaphore.
The previous implementation spawned N goroutines to group samples
together and would not start work until the semaphore unblocked.
While this didn't leak, it polluted the scheduling space.  Thusly,
the routine only starts after a semaphore has been acquired.
2013-05-21 16:11:35 +02:00
Julius Volz 5b105c77fc Repointerize fingerprints. 2013-05-21 14:28:14 +02:00
Matt T. Proud ec5b5bae28 Fuck you, Travis. 2013-05-21 09:42:00 +02:00
Matt T. Proud e5ac91222b Benchmark memory arena; simplify map generation.
The one-off keys have been replaced with ``model.LabelPair``, which is
indexable.  The performance impact is negligible, but it represents
a cognitive simplification.
2013-05-21 09:39:12 +02:00
juliusv 360477f66c Merge pull request #257 from prometheus/feature/better-memory-behaviors
Pointerize memorySeriesArena.
2013-05-16 07:36:40 -07:00
Matt T. Proud e1f20de2e9 Pointerize memorySeriesArena. 2013-05-16 17:09:28 +03:00
Matt T. Proud 8f4c7ece92 Destroy naked returns in half of corpus.
The use of naked return values is frowned upon.  This is the first
of two bulk updates to remove them.
2013-05-16 10:53:25 +03:00
Matt T. Proud 4e0c932a4f Simplify Encoder's encoding signature.
The reality is that if we ever try to encode a Protocol Buffer and it
fails, it's likely that such an error is ultimately not a runtime error
and should be fixed forthwith.  Thusly, we should rename
``Encoder.Encode`` to ``Encoder.MustEncode`` and drop the error return
value.
2013-05-16 00:54:18 +03:00
juliusv 516101f015 Merge pull request #250 from prometheus/refactor/drop-unused-storage-setting
Drop unused writeMemoryInterval
2013-05-14 08:45:59 -07:00
juliusv 9ff00b651d Merge pull request #251 from prometheus/fix/memory-metric-mutability
Fix GetMetricForFingerprint() metric mutability.
2013-05-14 08:12:45 -07:00
Bernerd Schaefer 63d9988b9c Drop unused writeMemoryInterval 2013-05-14 17:03:03 +02:00
Julius Volz 83c60ad43a Fix GetMetricForFingerprint() metric mutability.
Some users of GetMetricForFingerprint() end up modifying the returned metric
labelset. Since the memory storage's implementation of
GetMetricForFingerprint() returned a pointer to the metric (and maps are
reference types anyways), the external mutation propagated back into the memory
storage.

The fix is to make a copy of the metric before returning it.
2013-05-14 16:46:30 +02:00
Bernerd Schaefer 428d91c86f Rename test helper files to helpers_test.go
This ensures that these files are properly included only in testing.
2013-05-14 16:30:47 +02:00
juliusv 98e512d755 Merge pull request #246 from prometheus/fix/interval-value-extraction
Fix and optimize getValuesAtIntervalOp data extraction.
2013-05-14 05:55:22 -07:00
Julius Volz 71a3172abb Fix and optimize getValuesAtIntervalOp data extraction.
- only the data extracted in the last loop iteration of ExtractSamples() was
  emitted as output
- if e.g. op interval < sample interval, there were situations where the same
  sample was added multiple times to the output
2013-05-14 13:55:17 +02:00
Matt T. Proud 244a4a9cdb Update to go1.1.
This commit updates the documentation, Makefiles, formatting, and
code semantics to support the 1.1. runtime, which includes ...

1. ``make advice``,

2. ``make format``, and

3. ``go fix`` on various targets.
2013-05-14 12:39:08 +02:00
Matt T. Proud b224251981 Simplify compaction and expose database sizes.
This commit simplifies the way that compactions across a database's
keyspace occur due to reading the LevelDB internals. Secondarily it
introduces the database size estimation mechanisms.

Include database health and help interfaces.

Add database statistics; remove status goroutines.

This commit kills the use of Go routines to expose status throughout
the web components of Prometheus. It also dumps raw LevelDB status
on a separate /databases endpoint.
2013-05-14 12:29:53 +02:00
juliusv 92ad65ff13 Merge pull request #232 from prometheus/optimize/granular-storage-locking
Synchronous memory appends and more fine-grained storage locks.
2013-05-13 10:11:57 -07:00
Matt T. Proud 1f7f89b4e3 Simplify compaction and expose database sizes.
This commit simplifies the way that compactions across a database's
keyspace occur due to reading the LevelDB internals.  Secondarily it
introduces the database size estimation mechanisms.
2013-05-13 13:15:35 +02:00
Matt T. Proud d538b0382f Include long-tail data deletion mechanism.
This commit introduces the long-tail deletion mechanism, which will
automatically cull old sample values.  It is an acceptable
hold-over until we get a resampling pipeline implemented.

Kill legacy OS X documentation, too.
2013-05-13 10:54:36 +02:00