mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-28 06:59:40 -08:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity
702 commits 243 branches 444 tags 649 MiB
Commit graph

231 commits

Author SHA1 Message Date
Julius Volz d2da21121c Implement getValueRangeAtIntervalOp for faster range queries. 
This also short-circuits optimize() for now, since it is complex to implement
for the new operator, and ops generated by the query layer already fulfill the
needed invariants. We should still investigate later whether to completely
delete operator optimization code or extend it to support
getValueRangeAtIntervalOp operators.
 2013-06-26 18:10:36 +02:00
Julius Volz e7f049c85b Fix expunging of empty memory series (loop var pointerization bug) 2013-06-26 18:00:47 +02:00
Julius Volz baa5b07829 Fix condition for dropping empty memory series. 2013-06-25 17:57:35 +02:00
Matt T. Proud 30b1cf80b5 WIP - Snapshot of Moving to Client Model. 2013-06-25 15:52:42 +02:00
juliusv 42198c1f1c Merge pull request #311 from prometheus/fix/watermarking/on-first-write 
Ensure new metrics are watermarked early.
 2013-06-25 03:13:58 -07:00
Matt T. Proud b811ccc161 Disable paranoid checks and expose max FDs option. 
We shouldn't need paranoid checks now.  We also shouldn't need
too many FDs being open due to rule evaluator hitting in-memory
values stream.
 2013-06-24 12:10:14 +02:00
Matt T. Proud 4137c75523 Shrink default LRU cache sizes. 
Observing Prometheus in production confirms we can lower these values
safely.
 2013-06-24 12:09:16 +02:00
Matt T. Proud ecb9c7bb9d Code Review: Swap ordering of elements. 2013-06-21 21:17:50 +02:00
Matt T. Proud 5daa0a09ea Code Review: Swap ordering of watermark getting. 
A test for Julius.
 2013-06-21 18:34:08 +02:00
Matt T. Proud ee840904d2 Code Review: !Before -> After. 2013-06-21 18:26:40 +02:00
Matt T. Proud 2d5de99fbf Regard in-memory series as new. 
This commit ensures that series that exist only in-memory and not
on-disk are not regarded as too old for operation exclusion.
 2013-06-21 18:26:39 +02:00
Matt T. Proud 81c406630a Merge pull request #312 from prometheus/fix/sample-append-logging 
Log correct sample count when appending to disk.
 2013-06-21 08:55:51 -07:00
Matt T. Proud a1a23fbaf8 Ensure new metrics are watermarked early. 
With the checking of fingerprint freshness to cull stale metrics
from queries, we should write watermarks early to aid in more
accurate responses.
 2013-06-21 16:38:46 +02:00
Julius Volz ba8c122147 Log correct sample count when appending to disk. 2013-06-21 12:23:27 +02:00
Julius Volz f2b4067b7b Speedup and clean up operation optimization. 2013-06-20 03:01:13 +02:00
Julius Volz 008bc09da8 Move check for empty memory series to separate method. 2013-06-19 14:19:53 +02:00
Julius Volz 16364eda37 Drop empty series from memory after flushing. 2013-06-19 12:14:23 +02:00
Julius Volz 71199e2c93 Cache disk fingerprint->metric lookups in memory. 2013-06-18 14:08:58 +02:00
Matt T. Proud a73f061d3c Persist solely Protocol Buffers. 
An design question was open for me in the beginning was whether to
serialize other types to disk, but Protocol Buffers quickly won out,
which allows us to drop support for other types.  This is a good
start to cleaning up a lot of cruft in the storage stack and
can let us eventually decouple the various moving parts into
separate subsystems for easier reasoning.

This commit is not strictly required, but it is a start to making
the rest a lot more enjoyable to interact with.
 2013-06-08 11:02:35 +02:00
juliusv 95400cb785 Merge pull request #290 from prometheus/fix/go-vet 
Minor "go tool vet" cleanups
 2013-06-07 06:52:48 -07:00
Julius Volz 558281890b Minor "go tool vet" cleanups 2013-06-07 15:34:41 +02:00
juliusv 615972dd01 Merge pull request #288 from prometheus/fix/curator/fallthrough-compaction-ordering 
Fix fallthrough compaction value ordering.
 2013-06-07 05:46:15 -07:00
Matt T. Proud 86f63b078b Fix fallthrough compaction value ordering. 
We discovered a regression whereby data chunks could be appended out
of order if the fallthrough case was hit.
 2013-06-07 14:41:00 +02:00
Julius Volz 7b9ee95030 Minor LevelDB watermark handling cleanups. 2013-06-06 23:56:31 +02:00
Julius Volz 84741b227d Use LRU cache to avoid querying stale series. 2013-06-06 23:56:19 +02:00
Julius Volz f98853d7b7 Fix type error in watermark list handling. 2013-06-06 23:56:14 +02:00
Matt T. Proud ef1d5fd8a2 Introduce semaphores for tiered storage. 
This commit wraps the tiered storage access componnets in semaphores,
since we can handle several concurrent memory reads.
 2013-06-06 18:16:18 +02:00
Matt T. Proud 819045541e Code Review: Make double-drain a panic. 2013-06-06 12:40:06 +02:00
Matt T. Proud e217a9fb41 Race Work: Make memory arena locks more coarse. 
We can optimize these as needed later.
 2013-06-06 12:08:20 +02:00
Matt T. Proud beaaf386e7 Add storage state guards and transition callbacks. 
To ensure that we access tiered storage in the proper way, we have
guards now.
 2013-06-06 11:52:09 +02:00
Matt T. Proud abb5353ade Merge pull request #283 from prometheus/feature/storage/consult-watermark 
Include LRU cache for fingerprint watermarks.
 2013-06-06 02:33:45 -07:00
Matt T. Proud 2c3df44af6 Ensure database access waits until it is started. 
This commit introduces a channel message to ensure serving
state has been reached with the storage stack before anything attempts
to use it.
 2013-06-06 10:42:21 +02:00
Matt T. Proud cbe2f3a7b1 Include LRU cache for fingerprint watermarks. 2013-06-06 10:13:18 +02:00
Julius Volz 51689d965d Add debug timers to instant and range queries. 
This adds timers around several query-relevant code blocks. For now, the
query timer stats are only logged for queries initiated through the UI.
In other cases (rule evaluations), the stats are simply thrown away.

My hope is that this helps us understand where queries spend time,
especially in cases where they sometimes hang for unusual amounts of
time.
 2013-06-05 18:32:54 +02:00
Matt T. Proud 8339a189cb Code Review: Fix seriesPresent scope. 
The seriesPresent scope should be constrained to the scope of a
scanJob, since this is keyed to given series.
 2013-06-04 13:16:59 +02:00
Matt T. Proud fe41ce0b19 Conditionalize disk initializations. 
This commit conditionalizes the creation of the diskFrontier and
seriesFrontier along with the iterator such that they are provisioned
once something is actually required from disk.
 2013-06-04 12:53:57 +02:00
Julius Volz a8468a2e5e Fix reversed disk flush cutoff behavior. 2013-05-28 16:14:30 +02:00
Julius Volz eb1f956909 Revert "Revert "Ensure that all extracted samples are added to view."" 
This reverts commit 4b30fb86b4.
 2013-05-28 14:36:03 +02:00
Matt T. Proud 4b30fb86b4 Revert "Ensure that all extracted samples are added to view." 
This reverts commit 008314b5a8. By
running an automated git bisection described in
https://gist.github.com/matttproud-soundcloud/22a371a8d2cba382ea64
this commit was found.
 2013-05-23 13:36:22 +02:00
Julius Volz 750f862d9a Use GetBoundaryValues() for non-counter deltas. 2013-05-22 19:13:47 +02:00
Julius Volz f2b48b8c4a Make getValuesAtIntervalOp consume all chunk data in one pass. 
This is mainly a small performance improvement, since we skip past the last
extracted time immediately if it was also the last sample in the chunk, instead
of trying to extract non-existent values before the chunk end again and again
and only gradually approaching the end of the chunk.
 2013-05-22 18:14:45 +02:00
Julius Volz 83d60bed89 extractValuesAroundTime() code simplification. 2013-05-22 18:14:45 +02:00
Julius Volz 008314b5a8 Ensure that all extracted samples are added to view. 
The current behavior only adds those samples to the view that are extracted by
the last pass of the last processed op and throws other ones away. This is a
bug. We need to append all samples that are extracted by each op pass.

This also makes view.appendSamples() take an array of samples.
 2013-05-22 18:14:37 +02:00
Matt T. Proud b586801830 Code Review: Fix to-disk queue infinite growth. 
We discovered a bug while manually testing this branch on a live
instance, whereby the to-disk queue was never actually dumped to
disk.
 2013-05-22 17:59:53 +02:00
Matt T. Proud 285a8b701b Code Review: Extend lock. 2013-05-22 17:59:53 +02:00
Matt T. Proud 2526ab8c81 Code Review: Extend lock scope for appending. 2013-05-22 17:59:53 +02:00
Matt T. Proud f994482d15 Code Review: Avenues for future improvemnet noted. 2013-05-22 17:59:53 +02:00
Matt T. Proud 298a90c143 Code Review: Initial arena size name. 2013-05-22 17:59:53 +02:00
Matt T. Proud c07abf8521 Initial move away from skiplist. 2013-05-22 17:59:53 +02:00
Matt T. Proud 74a66fd938 Spawn grouping of fingerprints with free semaphore. 
The previous implementation spawned N goroutines to group samples
together and would not start work until the semaphore unblocked.
While this didn't leak, it polluted the scheduling space.  Thusly,
the routine only starts after a semaphore has been acquired.
 2013-05-21 16:11:35 +02:00