Fabian Reinartz
55a9b5428a
Add separate head mutex
...
Introduce a seperate mutex for the head blocks to avoid a race where
a post-compaction reload may run between switching the DB's base mutex
to create a new head block in an appender.
2017-03-04 17:41:01 +01:00
Fabian Reinartz
ffb24a98f4
Add missing unlock, run debug endpoint in benchmark
2017-03-02 21:54:02 +01:00
Fabian Reinartz
f734773214
Replace per-file locking with single PID lock file
...
File locks have a multitude of problems that make them hard to use
correctly. As they are just advisory, they are only meaningful to
prevent accidents like running the same process twice.
A simple PID file lock works reliably in those cases and is simpler.
2017-02-19 13:01:19 +01:00
Fabian Reinartz
afa084920c
Use Castagnoli polynomial for CRC32s; cache them
2017-02-15 21:13:17 -08:00
Fabian Reinartz
9c7a88223e
Add full encode/decode WAL cycle test
2017-02-14 21:55:50 -08:00
Fabian Reinartz
2c97428a79
Add basic WAL tests
2017-02-14 16:12:56 -08:00
Fabian Reinartz
84e8027a8e
Close consumed WAL segments, verify checksums on read
2017-02-14 00:42:06 -08:00
Fabian Reinartz
eeb03e97e2
Add headers and format flag for WAL segments
2017-02-14 00:24:53 -08:00
Fabian Reinartz
79944a5912
Break out WAL into segment files
2017-02-13 23:53:19 -08:00
Fabian Reinartz
9ddbd64d00
Move stats into meta.json file, cleanup, docs
2017-01-19 11:22:47 +01:00
Fabian Reinartz
343dd9d94c
Fix wrong byte size in WAL base ref
2017-01-17 08:40:31 +01:00
Fabian Reinartz
9cf49f68e9
wal: use larger buffer
2017-01-16 14:18:25 +01:00
Fabian Reinartz
1c80c33e72
Fix bug of unsorted postings lists being created
...
The former approach created unordered postings list by either
map iteration of new series being unsorted (fixable) or concurrent
writers creating new series interleaved.
We switch back to generating ephemeral references for a single batch.
Newly created series have to be re-set upon the next insert.
2017-01-13 16:22:20 +01:00
Fabian Reinartz
0dffd52238
Use page writer in compaction
2017-01-09 18:47:43 +01:00
Fabian Reinartz
54f5027406
Put WAL lock down into encoder
2017-01-06 18:36:42 +01:00
Fabian Reinartz
300f4e2abf
Use separate lock for series creation
...
This uses the head block's own lock to only lock if new series were
encountered.
In the general append case we just need to hold a
2017-01-06 18:10:50 +01:00
Fabian Reinartz
71efd2e08d
Periodically fsync WAL, make head cut async
2017-01-06 15:18:06 +01:00
Fabian Reinartz
2eb544c98e
Change file names and maker parsing safer
2017-01-06 13:13:22 +01:00
Fabian Reinartz
937cdb579c
Switch to sequential block names
...
This changes block directory names from the int64 timestamp
to sequential numbering.
2017-01-06 10:45:03 +01:00
Fabian Reinartz
dbd2b21d2e
Make persistence atomic
2017-01-02 14:41:13 +01:00
Fabian Reinartz
62b8ded0a5
Rename iterator value getters to At()
2017-01-02 13:33:06 +01:00
Fabian Reinartz
7280533c42
Add basic shard metrics
2016-12-31 09:48:49 +01:00
Fabian Reinartz
a009247ab7
Add matching of empty label
2016-12-30 19:36:28 +01:00
Fabian Reinartz
8aba95048a
Rename to OpenHeadBlock
2016-12-22 20:00:24 +01:00
Fabian Reinartz
e7edae39b2
Use page buffered writer for WAL
2016-12-22 16:14:34 +01:00
Fabian Reinartz
9c6a72aadd
Load head with WALs correctly
2016-12-22 15:54:39 +01:00
Fabian Reinartz
1dde3b6d31
Add WAL decoder+loading and benchmarks
2016-12-22 15:18:33 +01:00
Fabian Reinartz
0b8c77361e
Add initial WAL writing
2016-12-22 12:05:24 +01:00