* tsdb: Bug fix for further continued after crash deletions; added more tests.
Additionally: Added log line for block removal.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Addressed comment.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
## Changes:
* Rename dir when deleting
* Ignoring blocks with broken meta.json on start (we do that on reload)
* Compactor writes <ulid>.tmp-for-creation blocks instead of just .tmp
* Delete tmp-for-creation and tmp-for-deletion blocks during DB open.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
I did not want to move those in previous PR to make it easier to review. Now small cleanup time for readability. (:
## Changes
* Merge series goes to `storage/merge.go` leaving `fanout.go` for just fanout code.
* Moved `fanout test` code from weird separate package to storage.
* Unskiped one test: TestFanout_SelectSorted/chunk_querier
* Moved block series set codes responsible for querying blocks to `querier.go` from `compact.go`
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* tsdb: Added ChunkQueryable implementations to db; unified compactor, querier and fanout block iterating.
Chained to https://github.com/prometheus/prometheus/pull/7059
* NewMerge(Chunk)Querier now takies multiple primaries allowing tsdb DB code to use it.
* Added single SeriesEntry / ChunkEntry for all series implementations.
* Unified all vertical, and non vertical for compact and querying to single
merge series / chunk sets by reusing VerticalSeriesMergeFunc for overlapping algorithm (same logic as before)
* Added block (Base/Chunk/)Querier for block querying. We then use populateAndTomb(Base/Chunk/) to iterate over chunks or samples.
* Refactored endpoint tests and querier tests to include subtests.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Addressed comments from Brian and Beorn.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed snapshot test and added chunk iterator support for DBReadOnly.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed race when iterating over Ats first.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed tests.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed populate block tests.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed endpoints test.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed test.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Added test & fixed case of head open chunk.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed DBReadOnly tests and bug producing 1 sample chunks.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Added cases for partial block overlap for multiple full chunks.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Added extra tests for chunk meta after compaction.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed small vertical merge bug and added more tests for that.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* storage: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* tsdb: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* web: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* notifier: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* cmd: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* scripts: Verify that we are not using restricted packages
It checks that we are not directly importing 'sync/atomic'.
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* Reorganise imports in blocks
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* notifier/test: Apply PR suggestions
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* storage/remote: avoid storing references on newEntry
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* Revert "scripts: Verify that we are not using restricted packages"
This reverts commit 278d32748e.
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* web: Group imports accordingly
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* tsdb/chunks: Replace sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* tsdb/heaad: Replace sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* vendor: Make go.uber.org/atomic a direct dependency
There is no modifications to go.sum and vendor/ because
it was already vendored.
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* tsdb: Remove comments referring to the sync/atomic alignment bug
Related: https://golang.org/pkg/sync/atomic/#pkg-note-BUG
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* no panic the head memseries has chunks in it
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
* fix a panic when querying after a wal corruption.
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
* review nits
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
* Add test for reading the data after a wal corruption.
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
Update tsdb/db_test.go
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Update tsdb/db_test.go
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
* spellings
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
* Fix race during head compaction
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Comment out the test
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Skip test instead of commenting it out
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Optimized label regex matcher with literal prefix and/or suffix
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Added license
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Added more tests cases with newlines
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Restored deleted test
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Fixed nits introduced by https://github.com/prometheus/prometheus/pull/7334
* Added ChunkQueryable implementation to fanout and readyStorage.
* Added more comments.
* Changed NewVerticalChunkSeriesMerger to CompactingChunkSeriesMerger, removed tiny interface by reusing VerticalSeriesMergeFunc for overlapping algorithm for
both chunks and series, for both querying and compacting (!) + made sure duplicates are merged.
* Added ErrChunkSeriesSet
* Added Samples interface for seamless []promb.Sample to []tsdbutil.Sample conversion.
* Deprecating non chunks serieset based StreamChunkedReadResponses, added chunk one.
* Improved tests.
* Split remote client into Write (old storage) and read.
* Queryable client is now SampleAndChunkQueryable. Since we cannot use nice QueryableFunc I moved
all config based options to sampleAndChunkQueryableClient to aboid boilerplate.
In next commit: Changes for TSDB.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed bstream test license
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Simplified bstreamReader.loadNextBuffer()
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Fixed date in license
Signed-off-by: Marco Pracucci <marco@pracucci.com>
