mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-26 22:19:40 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
883 commits 243 branches 444 tags 649 MiB
Commit graph

120 commits

Author SHA1 Message Date
Fabian Reinartz 683b4c8c2a Add list of source head blocks to meta.json 2017-06-07 11:06:45 +02:00
Goutham Veeramachaneni 261cd9f393
Incorporate feedback. 
Move back to {Enable, Disable}Compactions.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-06 20:23:20 +05:30
Goutham Veeramachaneni a110a64abd
Add full Snapshot support 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-06 18:15:54 +05:30
Goutham Veeramachaneni a1c8425357
Initial implementation of HeadBlock Snapshots 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-05 13:48:31 +05:30
Goutham Veeramachaneni 29c73f05f2
Make sure that mint and maxt are not modified. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-27 21:59:49 +05:30
Goutham Veeramachaneni 44e9ae38b5
Incorporate PR feedback. 
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-26 21:26:31 +05:30
Goutham Veeramachaneni 6febabeb28
Final delete fixes. 
* Make sure no reads happen on the block when delete is in progress.
* Fix bugs in compaction.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-26 16:31:45 +05:30
Goutham Veeramachaneni f29fb62fba
Make TombstoneReader a Getter. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-24 11:24:24 +05:30
Goutham Veeramachaneni 31cf939448
Add NumTombstones to BlockMeta. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-23 17:37:04 +05:30
Goutham Veeramachaneni 3eb4119ab1
Make HeadBlock use WAL. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-23 16:15:16 +05:30
Goutham Veeramachaneni 244b73fce1
Rename for clarity and consistency. 
Misc. changes for code cleanliness.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-22 16:42:36 +05:30
Goutham Veeramachaneni 8434019ad9
Merge branch 'master' into deletes-1 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-22 12:58:38 +05:30
Goutham Veeramachaneni 662d8173fe
Make Appends after Delete visible. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-22 11:28:24 +05:30
Goutham Veeramachaneni 0b70333ef6
Add tests for tombstones. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-21 23:20:05 +05:30
Goutham Veeramachaneni 3dbb400bef
Initial implementation of Delete on DB. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-20 00:35:50 +05:30
Goutham Veeramachaneni 45d3db4e9e
Use a *mapTombstoneReader instead of map 
We need to recalculate the sorted ref list everytime we make a
Tombstones() call. This avoids that.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-19 11:56:37 +05:30
Fabian Reinartz 39df7e2bba Switch blocks to ULID directories, drop sequenc numbers 2017-05-18 16:09:30 +02:00
Goutham Veeramachaneni 34a86af3c6
Move tombstones to their own thing. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-17 08:36:56 +05:30
Goutham Veeramachaneni cea3c88f17
Add Tombstones() method to Block. 
Also add Seek() to TombstoneReader

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-16 19:48:28 +05:30
Goutham Veeramachaneni 3de55171d3
Add tests for tombstones and deletedIterator 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-16 12:43:33 +05:30
Goutham Veeramachaneni 4f1d857590
Implement Delete on HeadBlock 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-15 23:28:14 +05:30
Goutham Veeramachaneni 5579efbd5b
Initial implentation of Deletes on persistedBlock 
Very much a WIP

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-14 14:36:26 +05:30
Fabian Reinartz 5534e6c53c Make HeadBlock impl public, make interface private 2017-05-12 16:34:41 +02:00
Fabian Reinartz 5b3cb11f3d Merge pull request #65 from prometheus/comment-fixes 
Minor comment fixes and additions.
 2017-04-28 16:07:16 +02:00
Fabian Reinartz 2c46815adb Merge pull request #66 from prometheus/fix-meta-error-handling 
Fix closing of meta file if meta encoding failed
 2017-04-28 16:06:42 +02:00
Julius Volz e374db22bc Remove unused skiplist types 2017-04-28 15:57:01 +02:00
Julius Volz 94f1ac599e Fix closing of meta file if meta encoding failed 2017-04-28 15:45:30 +02:00
Julius Volz 8d1fb4fa01 Minor comment fixes and additions. 2017-04-28 15:41:42 +02:00
Fabian Reinartz 778103b450 Add liecence file and headers 2017-04-10 20:59:45 +02:00
Fabian Reinartz 3635569257 Trigger reload correctly on interrupted compaction 2017-03-20 10:41:43 +01:00
Fabian Reinartz 2c999836fb Add Queryable interface to Block 
This adds the Queryable interface to the Block interface. Head and
persisted blocks now implement their own Querier() method and thus
isolate customization (e.g. remapPostings) more cleanly.
 2017-03-20 10:21:21 +01:00
Fabian Reinartz 11be2cc585 Add composed Block interfaces, remove head generation 
This adds more lower-leve interfaces which are used to compose
to different Block interfaces.
The DB only uses interfaces instead of explicit persistedBlock and
headBlock. The headBlock generation property is dropped as the use-case
can be implemented using block sequence numbers.
 2017-03-20 09:02:36 +01:00
Fabian Reinartz 2c3b56350a Write meta.json file atomically 2017-03-01 17:19:57 +01:00
Fabian Reinartz 306831f151 Add per-block state ULID 2017-02-27 10:46:15 +01:00
Fabian Reinartz 9a5dfadb09 Tweak IndexWriter interface, serialize directly into dir 2017-02-25 07:24:20 +01:00
Fabian Reinartz 78780cd2ba Segment chunk file 
This adds write path support for segmented chunk data files.
Files of 512MB are pre-allocated and written to. If the file size
is exceeded, the next file is started. On completion, files
are truncated to their final size.
 2017-02-23 10:50:22 +01:00
Fabian Reinartz f734773214 Replace per-file locking with single PID lock file 
File locks have a multitude of problems that make them hard to use
correctly. As they are just advisory, they are only meaningful to
prevent accidents like running the same process twice.
A simple PID file lock works reliably in those cases and is simpler.
 2017-02-19 13:01:19 +01:00
Fabian Reinartz a3d042b54e Support multiple chunk files in read path 2017-02-18 17:33:20 +01:00
Fabian Reinartz 30efe4a58c Support writing to multiple head blocks 
This is an initial (and hacky) first pass on allowing
appending to multiple blocks simultaniously to avoid
dropping samples right after cutting a new head block.
It's also required for cases like the PGW, where a scrape may
contain varying timestamps.
 2017-02-01 15:57:28 +01:00
Fabian Reinartz c20cc44b06 Add docs, write sequence number to meta.json 2017-01-29 08:11:47 +01:00
Fabian Reinartz 30d8866c13 Limit compaction range, make cut and compact concurrent 2017-01-20 07:58:19 +01:00
Fabian Reinartz 67d185ceb9 Compact based on compaction generation 2017-01-19 19:45:52 +01:00
Fabian Reinartz 472c618c39 Drop out-of-bound samples 2017-01-19 15:03:57 +01:00
Fabian Reinartz d4779b374c Properly track and write meta file 2017-01-19 14:01:38 +01:00
Fabian Reinartz 9ddbd64d00 Move stats into meta.json file, cleanup, docs 2017-01-19 11:22:47 +01:00
Fabian Reinartz e006bc6dc6 Improve error messages, create regular dir for block 2017-01-19 08:40:15 +01:00
Fabian Reinartz d2322f6095 Improve compaction processing 2017-01-18 06:18:32 +01:00
Fabian Reinartz c32a94d409 Unexport HeadBlock, export Block interface 2017-01-10 15:41:57 +01:00
Fabian Reinartz 1943f8d1bb Fix head block stats races 2017-01-07 18:02:17 +01:00
Fabian Reinartz 2eb544c98e Change file names and maker parsing safer 2017-01-06 13:13:22 +01:00
Fabian Reinartz 937cdb579c Switch to sequential block names 
This changes block directory names from the int64 timestamp
to sequential numbering.
 2017-01-06 10:45:03 +01:00
Fabian Reinartz ac49f8c15e Consolidate persistence and compaction 2017-01-03 16:49:37 +01:00
Fabian Reinartz e7f04d14d5 Lock mmapped files 2017-01-03 10:09:20 +01:00
Fabian Reinartz 91b65b55e7 Run persistence in separate goroutine 2017-01-02 22:24:35 +01:00
Fabian Reinartz beb842a856 Change block interface 
This changes the block interface to directly expose index
and series readers rather than the direct querier
 2017-01-02 11:12:28 +01:00
Fabian Reinartz 8aba95048a Rename to OpenHeadBlock 2016-12-22 20:00:24 +01:00
Fabian Reinartz 9c6a72aadd Load head with WALs correctly 2016-12-22 15:54:39 +01:00
Fabian Reinartz d9ca4b47f5 Fix offset errors, fix persisted postings order 2016-12-20 13:14:55 +01:00
Fabian Reinartz 00a503129b Use contextualized and traced errors in reader 2016-12-19 22:29:49 +01:00
Fabian Reinartz bd77103a49 Add stats serialization, load querier of all blocks 2016-12-15 16:14:33 +01:00
Fabian Reinartz 9ceed5378e Fix missing bound checks, off-by-ones, typos 2016-12-15 11:24:20 +01:00
Fabian Reinartz 9873e18b75 Add loading of persisted blocks 2016-12-15 08:36:07 +01:00
Fabian Reinartz 6eeb0ef01c Add queriers and partial mocks 2016-12-13 15:26:58 +01:00
Fabian Reinartz 14dbc59f2b cleanup and switching removal of unsafe calls. 2016-12-10 18:09:57 +01:00
Fabian Reinartz 40a451694f Refactor persistence into interfaces 2016-12-09 20:45:46 +01:00
Fabian Reinartz 62f9dc311c misc 2016-12-09 16:54:38 +01:00
Fabian Reinartz 74f8dfd95d Persist blocks periodically 2016-12-09 13:41:38 +01:00
Fabian Reinartz 0cf8bb9e53 Move sub-indexes into single index structure 2016-12-09 10:41:51 +01:00
Fabian Reinartz 8aa99a3ebd misc 2016-12-09 10:00:14 +01:00
Fabian Reinartz 2c34a15fe6 Add initial seriailization of block data 2016-12-08 17:43:10 +01:00