mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-11-09 23:24:05 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
537 commits 245 branches 436 tags 650 MiB
Commit graph

198 commits

Author SHA1 Message Date
Fabian Reinartz fb9da52b11 Add more verbose error handling for closing, reduce locking 
This commit introduces error returns in various places and is explicit
about closing persisted blocks.
{Index,Chunk,Tombstone}Readers are more consistent about their Close()
method. Whenever a reader is retrieved, the corresponding close method
must eventually be called. We use this to track pending readers against
persisted blocks.

Querier's against the DB no longer hold a read lock for their entire
lifecycle. This avoids long running queriers to starve new ones when we
have to acquire a write lock when reloading blocks.
 2017-10-10 12:13:37 +02:00
Fabian Reinartz 27f1b8aac3 Merge pull request #162 from BasPH/fsync-duration 
Instrument WAL fsync
 2017-10-05 08:18:36 +02:00
Bas Harenslak 5e1c258a98 Instrument WAL fsync 2017-10-04 22:17:02 +02:00
Goutham Veeramachaneni da565f975e Merge pull request #161 from prometheus/fileutil 
Remove dependency on etcd/pkg/fileutil
 2017-10-04 17:08:54 +05:30
Fabian Reinartz bbe72dccb9 Remove dependency on etcd/pkg/fileutil 2017-10-04 10:23:41 +02:00
Dimitrij Denissenko 83c984d9a3 Use configurable WAL flush inteval 2017-10-03 11:35:38 +01:00
Fabian Reinartz e19b648377 Merge pull request #155 from Gouthamve/level-log 
Add levels to all log lines.
 2017-10-02 16:34:49 +02:00
Goutham Veeramachaneni c35d3a65bd
Add levels to all log lines. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-09-28 12:49:34 +05:30
Thibault Chataigner 0ff8f91ec8 Add db.Head() method 
Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
 2017-09-25 16:45:24 +02:00
Thibault Chataigner 5ca18837e9 Make DB blocks list public. 
This enable computing the first timestamp in the DB
only accessing public interfaces.

Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
 2017-09-21 11:56:12 +02:00
Goutham Veeramachaneni 8919baef03
Expose NewIndexReader() and cleanups 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-09-13 13:47:20 +05:30
Fabian Reinartz f904cd385f Do not build a superflous 'all' postings 2017-09-08 18:41:43 +02:00
Fabian Reinartz 81222849bc Filter WAL data in Head, misc fixes 2017-09-06 16:20:37 +02:00
Fabian Reinartz c36d574290 Replace single head lock with granular locks 
This adds various new locks to replace the single big lock on
the head. All parts now must be COW as they may be held by clients
after initial retrieval.
Series by ID and hashes are now held in a stripe lock to reduce
contention and total holding time during GC. This should reduce
starvation of readers.
 2017-09-05 14:41:39 +02:00
Fabian Reinartz af2c2f9674 Re-enable head block delet test 2017-09-04 15:07:30 +02:00
Goutham Veeramachaneni 1698c516ad [WIP]: WAL implementation 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-09-04 14:52:40 +02:00
Fabian Reinartz 893b6ec506 Add tests for GC and chunk truncation 2017-09-01 14:38:49 +02:00
Fabian Reinartz 4cc37eecab Refactor and add tests for compactor 2017-09-01 11:50:58 +02:00
Fabian Reinartz 5cf2662074 Refactor WAL into Head and misc improvements 2017-09-01 11:50:58 +02:00
Fabian Reinartz 8209e3ec23 Add various metrics 2017-09-01 11:50:58 +02:00
Fabian Reinartz 3901b6e70b Remove multiple heads 
This changes the structure to a single WAL backed by a single head
block.
Parts of the head block can be compacted. This relieves us from any head
amangement and greatly simplifies any consistency and isolation concerns
by just having a single head.
 2017-09-01 11:50:58 +02:00
Andreas Kohn 4003da0fdd
Log the directory when the WAL cannot be opened 
Fixes #125
 2017-08-22 11:17:07 +02:00
Fabian Reinartz 905af27cf9 Refactor compactor 2017-08-09 11:10:29 +02:00
Fabian Reinartz 66ff7b12e9 Pool Chunk objects during compaction 2017-08-08 17:35:34 +02:00
Fabian Reinartz fa04455cd7 Properly detect compactable heads 
The fullness of the wrong head was being tracked, causing wrong
lists of compactable heads.
 2017-08-03 13:58:11 +02:00
tianwenjie01 b64054365d Fix DefaultOptions 2017-07-18 17:42:58 +08:00
Fabian Reinartz 74f67e8271 Fix compacting disable/enable 
Enabling and disabling compaction no longer blocks are potentially
causes panics.
 2017-07-14 10:08:05 +02:00
Fabian Reinartz 3065be97d8 Fix and document locking order for DB 2017-07-14 09:00:22 +02:00
Fabian Reinartz 9c4235532e Fix compaction selection after creating new heads 
This fixes the case where between block creations no compaction
plans are ran. We were not compacting anything in these
cases since the on creation the most recent head block always had
a high timestamp of 0.
 2017-07-13 16:15:13 +02:00
Goutham Veeramachaneni 7b9c536883 Merge pull request #103 from Gouthamve/block-ranges 
Compaction implementation for block-ranges
 2017-07-12 17:32:49 +02:00
Goutham Veeramachaneni 9a2ab732b5 Choose ranges by retention and min-block-duration 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 17:30:34 +02:00
Goutham Veeramachaneni f1ae239c20 Persist the right MaxTime when snapshotting 
This is because we cut a new block from where the snapshotted block ends
if we restore from backups and highTimestamp would be where we should be
 starting from.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 13:48:13 +02:00
Goutham Veeramachaneni ce23adb579 Initial compaction implementation for block-ranges 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-07 16:32:18 +02:00
Fabian Reinartz 1e74c155eb Return empty string to signal non-caching 2017-06-26 14:58:00 +02:00
Fabian Reinartz 3410559c1b Compact head block early 
Let older head blocks be compacted once the newest once has samples at
50% of its total range. This allows the memory of the compacted blocks
to be released and garbage collected before a new head block gets
created. Thereby the number of head blocks is 1 or 2 instead of 2 or 3
and memory spikes are reduced.
 2017-06-26 08:52:59 +02:00
Fabian Reinartz 03f95ee096 Fix directory file descriptor leak. 2017-06-12 00:05:04 +02:00
Fabian Reinartz d4cb579443 Add DB.Dir method 2017-06-08 12:14:13 +02:00
Goutham Veeramachaneni ff4ccb6eb0
Remove unnecessary error from ToggleCompaction fns 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-06 23:45:23 +05:30
Goutham Veeramachaneni 261cd9f393
Incorporate feedback. 
Move back to {Enable, Disable}Compactions.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-06 20:23:20 +05:30
Goutham Veeramachaneni a110a64abd
Add full Snapshot support 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-06 18:15:54 +05:30
Goutham Veeramachaneni a1c8425357
Initial implementation of HeadBlock Snapshots 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-05 13:48:31 +05:30
Goutham Veeramachaneni 3a5ae6b1a4 Merge pull request #82 from Gouthamve/deletes-1 
Implement deletions
 2017-05-27 22:06:45 +05:30
Fabian Reinartz b3b9245ad6 Add and fix metrics 2017-05-26 15:13:03 +02:00
Goutham Veeramachaneni 6febabeb28
Final delete fixes. 
* Make sure no reads happen on the block when delete is in progress.
* Fix bugs in compaction.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-26 16:31:45 +05:30
Goutham Veeramachaneni 9bf7aa9af1
Misc. fixes incorporating feedback. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-23 18:13:30 +05:30
Goutham Veeramachaneni 8434019ad9
Merge branch 'master' into deletes-1 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-22 12:58:38 +05:30
Goutham Veeramachaneni 0b70333ef6
Add tests for tombstones. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-21 23:20:05 +05:30
Goutham Veeramachaneni d32eb25662
Synchronise deletes and compactions. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-20 13:21:10 +05:30
Goutham Veeramachaneni 3dbb400bef
Initial implementation of Delete on DB. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-20 00:35:50 +05:30
Fabian Reinartz debfe9b1e5 Simplify AddFast and avoid errgroup Commits in general 2017-05-18 17:17:56 +02:00