Commit graph

120 commits

Author SHA1 Message Date
Julius Volz 99dcbe0f94 Integrate memory and disk layers in view rendering. 2013-04-19 16:01:27 +02:00
Julius Volz 63625bd244 Make view use memory persistence, remove obsolete code.
This makes the memory persistence the backing store for views and
adjusts the MetricPersistence interface accordingly. It also removes
unused Get* method implementations from the LevelDB persistence so they
don't need to be adapted to the new interface. In the future, we should
rethink these interfaces.

All staleness and interpolation handling is now removed from the storage
layer and will be handled only by the query layer in the future.
2013-04-18 22:26:29 +02:00
Matt T. Proud d468271e2f Fix append queue telemetry and parameterize sizes.
The original append queue telemetry never worked, because it was
updated only upon the exit of the select statement, which would
usually liberate the queues of contents.  This has been fixed to
be reported arbitrarily.

The queue sizes are now parameterizable via flags.
2013-04-16 17:13:29 +02:00
Julius Volz 95b081f9bc Stop serving tiered storage after draining it. 2013-04-15 13:30:03 +02:00
Matt T. Proud a55602df4a Validate diskFrontier domain for series candidate.
It is the case with the benchmark tool that we thought that we
generated multiple series and saved them to the disk as such, when
in reality, we overwrote the fields of the outgoing metrics via
Go map reference behavior.  This was accidental.  In the course of
diagnosing this, a few errors were found:

1. ``newSeriesFrontier`` should check to see if the candidate fingerprint is within the given domain of the ``diskFrontier``.  If not, as the contract in the docstring stipulates, a ``nil`` ``seriesFrontier`` should be emitted.

2. In the interests of aiding debugging, the raw LevelDB ``levigoIterator`` type now includes a helpful forensics ``String()`` method.

This work produced additional cleanups:

1. ``Close() error`` with the storage stack is technically incorrect, since nowhere in the bowels of it does an error actually occur.  The interface has been simplified to remove this for now.
2013-04-09 11:47:16 +02:00
Matt T. Proud d79c932a8e Merge pull request #120 from prometheus/feature/storage/compaction
Spin up curator run in the tests.
2013-04-05 04:55:59 -07:00
Matt T. Proud c3e3460ca6 Spin up curator run in the tests.
After this commit, we'll need to add validations that it does the
desired work, which we presently know that it doesn't.  Given the
changes I made with a plethora of renamings, I want to commit this
now before it gets even larger.
2013-04-05 13:55:11 +02:00
Matt T. Proud 461da0b3a8 Merge pull request #117 from prometheus/feature/storage/compaction
Spin up storage layers for made fixtures.
2013-04-03 04:41:52 -07:00
Matt T. Proud d0ad6cbeaa Spin up storage layers for made fixtures. 2013-04-03 12:09:05 +02:00
Julius Volz c59f3fc538 Fix formatting in tiered_test.go. 2013-03-28 12:16:31 +01:00
juliusv 39826d7335 Merge pull request #107 from prometheus/julius-fix-get-fingerprints
Fix bug in GetFingerprintsForLabelSet().
2013-03-27 10:54:17 -07:00
Julius Volz 2668700e54 Fix bug in GetFingerprintsForLabelSet(). 2013-03-27 18:50:30 +01:00
Matt T. Proud c53a72a894 Test data for the curator. 2013-03-27 18:13:43 +01:00
Julius Volz 55ca65aa6e More userfriendly output when we fail to create the tiered storage. 2013-03-27 11:25:05 +01:00
Matt T. Proud c4e971d7d9 Merge pull request #101 from prometheus/refactor/test/directory-extraction
Create temporary directory handler.
2013-03-26 10:46:28 -07:00
Matt T. Proud b86b0ea41a Create temporary directory handler. 2013-03-26 18:09:25 +01:00
Julius Volz 8cf2af3923 Abort view job processing on timeout. 2013-03-26 17:18:51 +01:00
Julius Volz 2b8f0b2cc7 Constantize metric name label name. 2013-03-26 16:20:23 +01:00
Julius Volz e096896932 PR comment fixups. 2013-03-26 15:28:00 +01:00
Julius Volz dd67ab115b Change GetAllMetricNames() to GetAllValuesForLabel(). 2013-03-26 14:47:07 +01:00
Julius Volz 42bdf921d1 Fetch integrated memory/disk data for simple Get* functions. 2013-03-26 14:47:07 +01:00
Julius Volz 11bb94a7e5 Implement GetAllMetricNames() for memory storage. 2013-03-26 14:47:07 +01:00
Julius Volz 991dc68d78 Rename misnamed oldestSampleTimestamp variable. 2013-03-26 11:56:10 +01:00
Matt T. Proud 3e97a3630d Include nascent curator scaffolding.
The curator doesn't do anything yet; rather, this is the type
definition including the anciliary testing scaffold.

Improve Makefile and Git developer experience.

The top-level Makefile was a bit overloaded in terms of generation of
assets and their management.  This has been offloaded into separate
Makefiles.

The Git developer experience sucked due to lack of .gitignore
policies.

Also: Fix faulty skiplist naming from old merge.
2013-03-25 19:38:14 +01:00
Matt T. Proud b2e4c88b80 Wrap LevelDB iterator operations behind interface.
The LevelDB storage types return an interface type now that wraps
around the underlying iterator.  This both enhances testability but
improves upon, in my opinion, the interface design for the LevelDB
iterator.

Secondarily, the resource reaping behaviors for the LevelDB iterators
have been improved by dropping the externalized io.Closer object.

Finally, the iterator provisioning methods provide the option for
indicating whether one wants a snapshotted iterator or not.
2013-03-25 12:57:58 +01:00
Matt T. Proud 70448711ec Merge pull request #95 from prometheus/feature/persistence/batching
Several interface cleanups.
2013-03-24 00:19:46 -07:00
Matt T. Proud 8f6b55be71 Several interface cleanups.
- Kill Close in Persistent and document interface.
 - Extract batching behavior into interface.
 - Kill IteratorManager, which was used for unknown reasons.
2013-03-24 07:35:43 +01:00
Julius Volz a33d2726bc Mark range op as consumed if it receives no data points in range. 2013-03-22 11:50:02 +01:00
Julius Volz 3c9d6cb66c Add several needed persistence proxy methods to tiered storage. 2013-03-21 18:16:43 +01:00
Julius Volz 081d250929 Fix view's GetRangeValues() reverse iteration behavior. 2013-03-21 18:16:31 +01:00
Julius Volz 0be0aa59c2 Wait until storage is drained before closing the underlying leveldb. 2013-03-21 18:16:07 +01:00
Julius Volz becc278eb6 Fix two bugs in range op time advancement. 2013-03-21 18:15:52 +01:00
Matt T. Proud ceb6611957 Fix regression in subsequent range op. compactions.
We have an anomaly whereby subsequent range operations fail to be
compacted into one single range operation.  This fixes such
behavior.
2013-03-21 18:11:04 +01:00
Matt T. Proud 669abdfefe `make format` invocation. 2013-03-21 18:11:04 +01:00
Julius Volz bdb067b47f Implement remaining View Get* methods. 2013-03-21 18:11:04 +01:00
Julius Volz 1f42364733 Fix typo in comment. 2013-03-21 18:11:03 +01:00
Matt T. Proud 758a3f0764 Add documentation and cull junk. 2013-03-21 18:11:03 +01:00
Matt T. Proud bd8bb0edfd One additional reduction. 2013-03-21 18:11:03 +01:00
Matt T. Proud 73b463e814 Additional simplifications. 2013-03-21 18:11:03 +01:00
Matt T. Proud fd47ac570f Implied simplifications. 2013-03-21 18:11:03 +01:00
Matt T. Proud 51a0f21cf8 Interim documentation 2013-03-21 18:11:03 +01:00
Matt T. Proud b470f925b7 Extract rewriting of interval queries. 2013-03-21 18:11:03 +01:00
Matt T. Proud eb721fd220 Include note about greediest range. 2013-03-21 18:11:03 +01:00
Julius Volz e50de005f9 Populate metric in SampleSet returned from GetRangeValues() 2013-03-21 18:11:03 +01:00
Julius Volz 6001d22f87 Change Get* methods to receive fingerprints instead of metrics. 2013-03-21 18:11:03 +01:00
Julius Volz 95f8885c8a Adopt new ops sorting interface in view rendering. 2013-03-21 18:11:02 +01:00
Julius Volz 4d79dc3602 Replace renderView() by cleaner and more correct reimplementation. 2013-03-21 18:11:02 +01:00
Julius Volz e0dbc8c561 Fix edge cases in data extraction for point and interval ops. 2013-03-21 18:11:02 +01:00
Julius Volz a4361e4116 Rename extractSampleValue -> extractSampleValues. 2013-03-21 18:08:49 +01:00
Julius Volz 4e7db57e76 Fix iterator behavior in view.GetSampleAtTime() 2013-03-21 18:08:49 +01:00