Fabian Reinartz
778103b450
Add liecence file and headers
2017-04-10 20:59:45 +02:00
Fabian Reinartz
3635569257
Trigger reload correctly on interrupted compaction
2017-03-20 10:41:43 +01:00
Fabian Reinartz
2c999836fb
Add Queryable interface to Block
...
This adds the Queryable interface to the Block interface. Head and
persisted blocks now implement their own Querier() method and thus
isolate customization (e.g. remapPostings) more cleanly.
2017-03-20 10:21:21 +01:00
Fabian Reinartz
11be2cc585
Add composed Block interfaces, remove head generation
...
This adds more lower-leve interfaces which are used to compose
to different Block interfaces.
The DB only uses interfaces instead of explicit persistedBlock and
headBlock. The headBlock generation property is dropped as the use-case
can be implemented using block sequence numbers.
2017-03-20 09:02:36 +01:00
Fabian Reinartz
2c3b56350a
Write meta.json file atomically
2017-03-01 17:19:57 +01:00
Fabian Reinartz
306831f151
Add per-block state ULID
2017-02-27 10:46:15 +01:00
Fabian Reinartz
9a5dfadb09
Tweak IndexWriter interface, serialize directly into dir
2017-02-25 07:24:20 +01:00
Fabian Reinartz
78780cd2ba
Segment chunk file
...
This adds write path support for segmented chunk data files.
Files of 512MB are pre-allocated and written to. If the file size
is exceeded, the next file is started. On completion, files
are truncated to their final size.
2017-02-23 10:50:22 +01:00
Fabian Reinartz
f734773214
Replace per-file locking with single PID lock file
...
File locks have a multitude of problems that make them hard to use
correctly. As they are just advisory, they are only meaningful to
prevent accidents like running the same process twice.
A simple PID file lock works reliably in those cases and is simpler.
2017-02-19 13:01:19 +01:00
Fabian Reinartz
a3d042b54e
Support multiple chunk files in read path
2017-02-18 17:33:20 +01:00
Fabian Reinartz
30efe4a58c
Support writing to multiple head blocks
...
This is an initial (and hacky) first pass on allowing
appending to multiple blocks simultaniously to avoid
dropping samples right after cutting a new head block.
It's also required for cases like the PGW, where a scrape may
contain varying timestamps.
2017-02-01 15:57:28 +01:00
Fabian Reinartz
c20cc44b06
Add docs, write sequence number to meta.json
2017-01-29 08:11:47 +01:00
Fabian Reinartz
30d8866c13
Limit compaction range, make cut and compact concurrent
2017-01-20 07:58:19 +01:00
Fabian Reinartz
67d185ceb9
Compact based on compaction generation
2017-01-19 19:45:52 +01:00
Fabian Reinartz
472c618c39
Drop out-of-bound samples
2017-01-19 15:03:57 +01:00
Fabian Reinartz
d4779b374c
Properly track and write meta file
2017-01-19 14:01:38 +01:00
Fabian Reinartz
9ddbd64d00
Move stats into meta.json file, cleanup, docs
2017-01-19 11:22:47 +01:00
Fabian Reinartz
e006bc6dc6
Improve error messages, create regular dir for block
2017-01-19 08:40:15 +01:00
Fabian Reinartz
d2322f6095
Improve compaction processing
2017-01-18 06:18:32 +01:00
Fabian Reinartz
c32a94d409
Unexport HeadBlock, export Block interface
2017-01-10 15:41:57 +01:00
Fabian Reinartz
1943f8d1bb
Fix head block stats races
2017-01-07 18:02:17 +01:00
Fabian Reinartz
2eb544c98e
Change file names and maker parsing safer
2017-01-06 13:13:22 +01:00
Fabian Reinartz
937cdb579c
Switch to sequential block names
...
This changes block directory names from the int64 timestamp
to sequential numbering.
2017-01-06 10:45:03 +01:00
Fabian Reinartz
ac49f8c15e
Consolidate persistence and compaction
2017-01-03 16:49:37 +01:00
Fabian Reinartz
e7f04d14d5
Lock mmapped files
2017-01-03 10:09:20 +01:00
Fabian Reinartz
91b65b55e7
Run persistence in separate goroutine
2017-01-02 22:24:35 +01:00
Fabian Reinartz
beb842a856
Change block interface
...
This changes the block interface to directly expose index
and series readers rather than the direct querier
2017-01-02 11:12:28 +01:00
Fabian Reinartz
8aba95048a
Rename to OpenHeadBlock
2016-12-22 20:00:24 +01:00
Fabian Reinartz
9c6a72aadd
Load head with WALs correctly
2016-12-22 15:54:39 +01:00
Fabian Reinartz
d9ca4b47f5
Fix offset errors, fix persisted postings order
2016-12-20 13:14:55 +01:00
Fabian Reinartz
00a503129b
Use contextualized and traced errors in reader
2016-12-19 22:29:49 +01:00
Fabian Reinartz
bd77103a49
Add stats serialization, load querier of all blocks
2016-12-15 16:14:33 +01:00
Fabian Reinartz
9ceed5378e
Fix missing bound checks, off-by-ones, typos
2016-12-15 11:24:20 +01:00
Fabian Reinartz
9873e18b75
Add loading of persisted blocks
2016-12-15 08:36:07 +01:00
Fabian Reinartz
6eeb0ef01c
Add queriers and partial mocks
2016-12-13 15:26:58 +01:00
Fabian Reinartz
14dbc59f2b
cleanup and switching removal of unsafe calls.
2016-12-10 18:09:57 +01:00
Fabian Reinartz
40a451694f
Refactor persistence into interfaces
2016-12-09 20:45:46 +01:00
Fabian Reinartz
62f9dc311c
misc
2016-12-09 16:54:38 +01:00
Fabian Reinartz
74f8dfd95d
Persist blocks periodically
2016-12-09 13:41:38 +01:00
Fabian Reinartz
0cf8bb9e53
Move sub-indexes into single index structure
2016-12-09 10:41:51 +01:00
Fabian Reinartz
8aa99a3ebd
misc
2016-12-09 10:00:14 +01:00
Fabian Reinartz
2c34a15fe6
Add initial seriailization of block data
2016-12-08 17:43:10 +01:00