mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-27 14:39:40 -08:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity
466 commits 243 branches 444 tags 649 MiB
Commit graph

79 commits

Author SHA1 Message Date
Fabian Reinartz c36d574290 Replace single head lock with granular locks 
This adds various new locks to replace the single big lock on
the head. All parts now must be COW as they may be held by clients
after initial retrieval.
Series by ID and hashes are now held in a stripe lock to reduce
contention and total holding time during GC. This should reduce
starvation of readers.
 2017-09-05 14:41:39 +02:00
Fabian Reinartz 1ddedf2b30 Change series ID from uint32 to uint64 2017-09-04 16:08:38 +02:00
Fabian Reinartz 9f41d9fd3c Instrument chunks on level 1 compactions 2017-09-01 16:10:10 +02:00
Fabian Reinartz 4cc37eecab Refactor and add tests for compactor 2017-09-01 11:50:58 +02:00
Fabian Reinartz 3901b6e70b Remove multiple heads 
This changes the structure to a single WAL backed by a single head
block.
Parts of the head block can be compacted. This relieves us from any head
amangement and greatly simplifies any consistency and isolation concerns
by just having a single head.
 2017-09-01 11:50:58 +02:00
Goutham Veeramachaneni 7438ed7035 Expose Intervals type for use by TombstoneReader. 
TombstoneReader is exposed but Intervals is not.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-08-25 16:06:36 +05:30
Goutham Veeramachaneni 178f840295 Fix panic caused by 0 division 
Introduced in #108

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-08-18 07:45:29 +02:00
Fabian Reinartz 905af27cf9 Refactor compactor 2017-08-09 11:10:29 +02:00
Fabian Reinartz 66ff7b12e9 Pool Chunk objects during compaction 2017-08-08 17:35:34 +02:00
Fabian Reinartz 2644c8665c Don't allocate ChunkMetas, reuse postings slices 2017-08-06 20:41:24 +02:00
Fabian Reinartz 96d7f540d4 Persist series without allocating the full set 
Change index persistence for series to not be accumulated in memory
before being written as one large batch. `Labels` and `ChunkMeta`
objects are reused.
This cuts down memory spikes during compaction of multiple blocks
significantly.

As part of the the Index{Reader,Writer} now have an explicit notion of
symbols and series must be inserted in order.
 2017-08-06 12:06:41 +02:00
Fabian Reinartz 3951d8cc29 Fix selection of compactable blocks 2017-08-04 12:02:34 +02:00
Fabian Reinartz 5d7b5994d6 Fix compaction range selection 2017-07-13 16:13:59 +02:00
Goutham Veeramachaneni 401bdfd4a6 Make Chunks return the number of samples inside 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 18:45:12 +02:00
Goutham Veeramachaneni 1263a68875 Recompact blocks with large number of tombstones 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 18:27:31 +02:00
Goutham Veeramachaneni 9a2ab732b5 Choose ranges by retention and min-block-duration 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 17:30:34 +02:00
Goutham Veeramachaneni ce23adb579 Initial compaction implementation for block-ranges 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-07 16:32:18 +02:00
Fabian Reinartz 03f95ee096 Fix directory file descriptor leak. 2017-06-12 00:05:04 +02:00
Fabian Reinartz 05e411a8eb Improve heuristic to spread chunks across block 2017-06-08 11:30:32 +02:00
Fabian Reinartz f006e2d1ab Merge pull request #92 from prometheus/sources 
Add list of source head blocks to meta.json
 2017-06-07 11:50:44 +02:00
Fabian Reinartz 683b4c8c2a Add list of source head blocks to meta.json 2017-06-07 11:06:45 +02:00
Goutham Veeramachaneni a1c8425357
Initial implementation of HeadBlock Snapshots 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-05 13:48:31 +05:30
Goutham Veeramachaneni 3a5ae6b1a4 Merge pull request #82 from Gouthamve/deletes-1 
Implement deletions
 2017-05-27 22:06:45 +05:30
Goutham Veeramachaneni 44e9ae38b5
Incorporate PR feedback. 
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-26 21:26:31 +05:30
Fabian Reinartz b3b9245ad6 Add and fix metrics 2017-05-26 15:13:03 +02:00
Goutham Veeramachaneni 6febabeb28
Final delete fixes. 
* Make sure no reads happen on the block when delete is in progress.
* Fix bugs in compaction.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-26 16:31:45 +05:30
Goutham Veeramachaneni f29fb62fba
Make TombstoneReader a Getter. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-24 11:24:24 +05:30
Goutham Veeramachaneni 244b73fce1
Rename for clarity and consistency. 
Misc. changes for code cleanliness.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-22 16:42:36 +05:30
Goutham Veeramachaneni 8434019ad9
Merge branch 'master' into deletes-1 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-22 12:58:38 +05:30
Goutham Veeramachaneni 0b70333ef6
Add tests for tombstones. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-21 23:20:05 +05:30
Goutham Veeramachaneni 3dbb400bef
Initial implementation of Delete on DB. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-20 00:35:50 +05:30
Fabian Reinartz 7e6961b4ac Fix import and var name 2017-05-18 17:30:52 +02:00
Fabian Reinartz d2673d8659 Completely drop empty blocks 2017-05-18 16:12:17 +02:00
Fabian Reinartz 39df7e2bba Switch blocks to ULID directories, drop sequenc numbers 2017-05-18 16:09:30 +02:00
Goutham Veeramachaneni 22c1b5b492
Make SeriesSets use tombstones. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-17 14:49:42 +05:30
Goutham Veeramachaneni 5579efbd5b
Initial implentation of Deletes on persistedBlock 
Very much a WIP

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-14 14:36:26 +05:30
Fabian Reinartz 778103b450 Add liecence file and headers 2017-04-10 20:59:45 +02:00
Fabian Reinartz 10c7c9acbe Adjust import names to new repository organisation 2017-04-04 11:27:26 +02:00
Fabian Reinartz 789e8224ff Fix wrong comparison in head block resorting 2017-03-21 12:12:33 +01:00
Fabian Reinartz 9c93f8f2aa Fix various races 
This fixes different race condition encoutnered when running Prometheus.
It reduces the overall performance in the synthetic benchmark a fair bit
but has no indiciations of impacting a real-world setup notably.
 2017-03-20 14:45:27 +01:00
Fabian Reinartz 3635569257 Trigger reload correctly on interrupted compaction 2017-03-20 10:41:43 +01:00
Fabian Reinartz a8e8903350 Use ChunkMeta references for clarity 
This has been a common source of hard to debug issues. Its a premature
and unbenchmarked optimization and semantically, we want ChunkMetas to
be references in all changed cases.
 2017-03-14 15:40:16 +01:00
Fabian Reinartz 04783831e2 Correctly close compacted blocks. 2017-03-06 09:33:55 +01:00
Fabian Reinartz 21b97d1e04 Fix compaction selection procedure 2017-03-02 15:10:13 +01:00
Fabian Reinartz 327e07e8d0 Properly sync moves and removes in compactor 2017-03-02 14:35:06 +01:00
Fabian Reinartz 92120448c2 Properly cleanup compacted dirs, fixes, docs 2017-03-02 14:32:09 +01:00
Fabian Reinartz 2c3e778d90 Compactor interface 2017-03-02 09:17:30 +01:00
Fabian Reinartz 2c3b56350a Write meta.json file atomically 2017-03-01 17:19:57 +01:00
Fabian Reinartz db5c88ea9a Misc compaction fixes 2017-02-28 08:40:51 +01:00
Fabian Reinartz 306831f151 Add per-block state ULID 2017-02-27 10:46:15 +01:00