After this commit, we'll need to add validations that it does the
desired work, which we presently know that it doesn't. Given the
changes I made with a plethora of renamings, I want to commit this
now before it gets even larger.
The curator doesn't do anything yet; rather, this is the type
definition including the anciliary testing scaffold.
Improve Makefile and Git developer experience.
The top-level Makefile was a bit overloaded in terms of generation of
assets and their management. This has been offloaded into separate
Makefiles.
The Git developer experience sucked due to lack of .gitignore
policies.
Also: Fix faulty skiplist naming from old merge.
The LevelDB storage types return an interface type now that wraps
around the underlying iterator. This both enhances testability but
improves upon, in my opinion, the interface design for the LevelDB
iterator.
Secondarily, the resource reaping behaviors for the LevelDB iterators
have been improved by dropping the externalized io.Closer object.
Finally, the iterator provisioning methods provide the option for
indicating whether one wants a snapshotted iterator or not.
- Kill Close in Persistent and document interface.
- Extract batching behavior into interface.
- Kill IteratorManager, which was used for unknown reasons.
The LevelDB index retrievals could be repeated in a given operation
batch if multiple queued mutations affect the same (Label Name) singles
and (Label Name, Label Value) doubles. This is wasteful and
inefficient, as a single retrieval suffices. Thusly this commit
retrieves the canonical index mappings if the said mapping has not
been looked up in a given batch.
Presently our use of LevelDB and its operating modes are hardcoded
into the storage stack. This pull request decouples this and
re-exposes this through flags. We can now perform benchmarking
and remedial tuning.
client_golang was updated to support full label-oriented telemetry,
which introduced interface incompatibilities with the previous
version of Prometheus. To alleviate this, a general fetching and
processing dispatching system has been created, which discriminates
and processes according to the version of input.
This implementation needs to be cleaned up considerably, but it
should hopefully provide some breathing room. Ultimately this will
be extracted out of the storage layer---specifically the LevelDB
hierarchy---to interface with a generic set of abstractions for a
storage system, but this will be where it remains for now.
The LevelDB stochastic tests could take a long time.
Original:
```
Matt:prometheus mtp$ make test
make -C model
protoc --go_out=generated/ data.proto
go build ./...
go test ./...
warning: building out-of-date packages:
github.com/matttproud/golang_instrumentation/maths
github.com/matttproud/golang_instrumentation/utility
github.com/matttproud/golang_instrumentation/metrics
github.com/matttproud/golang_instrumentation
installing these packages with 'go test -i ./...' will speed future tests.
? github.com/matttproud/prometheus [no test files]
? github.com/matttproud/prometheus/coding [no test files]
ok github.com/matttproud/prometheus/coding/indexable 0.012s
? github.com/matttproud/prometheus/model [no test files]
? github.com/matttproud/prometheus/model/generated [no test files]
? github.com/matttproud/prometheus/retrieval [no test files]
? github.com/matttproud/prometheus/storage/metric [no test files]
ok github.com/matttproud/prometheus/storage/metric/leveldb 70.800s
? github.com/matttproud/prometheus/storage/raw [no test files]
? github.com/matttproud/prometheus/storage/raw/index [no test files]
ok github.com/matttproud/prometheus/storage/raw/index/leveldb 0.012s
ok github.com/matttproud/prometheus/storage/raw/leveldb 0.016s
ok github.com/matttproud/prometheus/utility 0.012s
? github.com/matttproud/prometheus/utility/test [no test files]
```
to
```
Matt:prometheus mtp$ make test
make -C model
protoc --go_out=generated/ data.proto
go build ./...
go test ./...
warning: building out-of-date packages:
github.com/matttproud/golang_instrumentation/maths
github.com/matttproud/golang_instrumentation/utility
github.com/matttproud/golang_instrumentation/metrics
github.com/matttproud/golang_instrumentation
installing these packages with 'go test -i ./...' will speed future tests.
? github.com/matttproud/prometheus [no test files]
? github.com/matttproud/prometheus/coding [no test files]
ok github.com/matttproud/prometheus/coding/indexable 0.011s
? github.com/matttproud/prometheus/model [no test files]
? github.com/matttproud/prometheus/model/generated [no test files]
? github.com/matttproud/prometheus/retrieval [no test files]
? github.com/matttproud/prometheus/storage/metric [no test files]
ok github.com/matttproud/prometheus/storage/metric/leveldb 2.158s
? github.com/matttproud/prometheus/storage/raw [no test files]
? github.com/matttproud/prometheus/storage/raw/index [no test files]
ok github.com/matttproud/prometheus/storage/raw/index/leveldb 0.013s
ok github.com/matttproud/prometheus/storage/raw/leveldb 0.013s
ok github.com/matttproud/prometheus/utility 0.013s
? github.com/matttproud/prometheus/utility/test [no test files]
```
Kill LevelDB watermarks due to redundancy.
General interface documentation has begun.
Creating custom types for the model to prevent errors down the
road.
Renaming of components for easier comprehension.
Exposition of interface in LevelDB.
Slew of simple refactorings.
