Commit graph

1 commit

Author SHA1 Message Date
Mauro Stettler cc9ddf50c9
Write m-map chunks in async via a queue while predicting the m-map chunk refs (#57)
* write chunks via queue, predicting chunkrefs

* fixes

* linter

* cleanup

* better test coverage

* deduplicate operations when cutting file

* better comments

* fix race

* improvements based on PR feedback

* comments

* concurrent correctness test of write queue

* linter

* use isStarted flag in chunk write queue

* separate mutex for isStarted

* fixing tests

* fix race in test

* PR feedback

* add license headers

* PR feedback

* pr feedback

* dont recreate workerCtrl channel

* address PR feedback in high concurrency read and write test

* refactor the high concurrency write queue test

* pr feedback

* use require.Eventually

* typo

* Fixing comment

Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>

* use buffered chan for chunk write queue

* fix races

* address feedback on PR

* better comment

* less type casts

* dont reeimplement varint

* move mtx above property it protects

* improve tests by using wg instead of inspecting queue

* add comment

* add comment

* writeChunkF comment

* rename isStarted -> isRunning

* prevent panic when double stopping

* fix variable name in comment

* delete outdated comment

* naming and comment fixes in TestChunkWriteQueue_WrappingAroundSizeLimit

* use require.False instead of require.Equal(t, false

* always enable chunk write queue

* dont call callback with lock held

* undo rename of curFileSequence to curFileSeq

* fix accidental change

* fix comment

* Better syntax

Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>

* Better wording in comment

Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>

* better comments to explain the use of coordinator chans in test

* fix test which broke to the async writing of chunks

* undo previous renaming of shadowing variable

* rename var threadID to workerID

* rename variables based on PR feedback

* rename pos -> ts

* better implementation of whileNotCanceled

* update querySeriesRef before using it

* rename labels -> lbls

* initialize the chunk write queue operations metric with all possible labels

* dont return error from CutNewFile

* recreate chunk ref map regularly to free

* limit how often we recreate the chunk ref map

* fix typo in comment

* better comment

Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>
2021-12-08 13:14:26 +05:30