mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-12-26 06:04:05 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
1062 commits 243 branches 444 tags 649 MiB
Commit graph

36 commits

Author SHA1 Message Date
Krasi Georgiev e809cb477d
Add e2e benchmarks for head and block Queries (#467) 
* Add e2e had and block query benchmarks

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-06-07 15:41:44 +02:00
naivewong 13c80a5979 Optimize queries using regex matchers for set lookups (#602) 
* Original version of the set optimization

Signed-off-by: naivewong <867245430@qq.com>

* simple set matcher

Signed-off-by: naivewong <867245430@qq.com>

* simple set matcher

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* add benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* use genSeries from #467

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>
 2019-05-27 16:54:46 +05:30
zhulongcheng 19d402d154 cleanup tmp files (#570) 
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-04-26 11:27:36 +03:00
zhulongcheng 4d03c70800 Verify chunk format version (#544) 
Verify chunk format version.
 2019-03-24 22:33:08 +02:00
Krasi Georgiev c3ffdf1a99
Test createBlock and check all os.RemoveAll in the tests for errors. (#549) 
Testing that createBlock creates blocks that can be opened.

and checking the os.RemoveAll for errors will catch errors for un-closed files under windows.

Many missing .Close() calls were added for fixing failing os.RemoveAll

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-03-19 15:31:57 +02:00
Ganesh Vernekar c59ed492b2 Vertical query merging and compaction (#370) 
* Vertical series iterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Select overlapped blocks first in compactor Plan()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added vertical compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Code cleanup and comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add benchmark for compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Perform vertical compaction only when blocks are overlapping.

Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for vertical compaction

* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for query iterator and seek for non overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Vertical query merge only for overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Simplify logging in Compact(...)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Calculate overlapping inside populateBlock

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* MinTime and MaxTime for BlockReader.

Using this to find overlapping blocks in populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Sort blocks w.r.t. MinTime in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping in LeveledCompactor.write() instead of returning bool

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping inside LeveledCompactor.populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor createBlock to take optional []Series

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* review1

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* Updated CHANGELOG and minor nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor iterator and seek benchmarks for Querier.

Also has as overlapping blocks.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Additional test case

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Split genSeries into genSeries and populateSeries

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Check error in benchmark

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Warn about overlapping blocks in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-14 14:29:41 +01:00
Krasi Georgiev ce4a2083fb nit 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-04 11:15:08 +02:00
Krasi Georgiev 00d5e19baf Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 13:47:50 +02:00
Krasi Georgiev dac2b97dfd
make createBlock more generic so it can be used in other tests. (#489) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-28 14:24:49 +03:00
Krasi Georgiev cc277e398b
fix the refs logic for the addFast path for createBlock (#504) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-21 14:56:26 +03:00
Krasi Georgiev 9874377ead nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-18 19:22:45 +02:00
Krasi Georgiev 921f82cfc1 WIP 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-18 18:58:17 +02:00
Krasi Georgiev 0b31b8af88 Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-18 16:58:18 +02:00
Ganesh Vernekar 1a9d08adc5 Don't write empty blocks (#374) 
* Dont write empty blocks when a compaction results in a block with no samples.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-01-18 11:35:16 +03:00
mknapphrt ebf5d74325 Added storage size based retention method and new metrics (#343) 
Added methods needed to retain data based on a byte limitation rather than time. Limitation is only applied if the flag is set (defaults to 0). Both blocks that are older than the retention period and the blocks that make the size of the storage too large are removed.

2 new metrics for keeping track of the size of the local storage folder and the amount of times data has been deleted because the size restriction was exceeded.
Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>
 2019-01-16 13:03:52 +03:00
Krasi Georgiev eb6586f513
rename createPopulatedBlock to createBlock and use it instead ot createEmptyBlock (#488) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-29 14:20:51 +03:00
Krasi Georgiev a90a7195d4
createPopulatedBlock returns the block dir instead of the block (#487) 
It is easy to forget to close the block returned by createPopulatedBlock
which causes failures for windows so instead it returns the block dir
and which can be used by OpenBlock explicitly.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-29 10:23:56 +03:00
Krasi Georgiev bd5ccee5c1 use context to cancel compactions 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-10 12:48:30 +02:00
Krasi Georgiev bac9cbed2e
no overlapping on compaction when an existing block is not within default boundaries. (#461) 
closes https://github.com/prometheus/prometheus/issues/4643

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-04 13:30:49 +03:00
Krasi Georgiev 5a9ddeecef
fix lint errors (#439) 
unexported NewMemTombstones as this returns unexported memTombstones
type which will not be shows in godoc.
Added missing comments for exported methods.
Removed unused RecordLogger,RecordReader interfaces.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-11-14 18:40:01 +02:00
Chris Marchbanks f4afc7dff2 Add benchmark for querying a persisted block (#425) 
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
 2018-10-24 00:35:52 +03:00
codwu 84a45cb79a add rwmutex to prevent concurrent map read when delete series 
Signed-off-by: codwu <wuhan9087@163.com>
 2018-06-08 19:52:01 +08:00
Fabian Reinartz 76c1b2cdb6 Ensure correct block order on reload 
Due to a regression blocks were no longer ordered by time before
being stored in memory. This made data intermittently become unavailable
for queries.

Signed-off-by: Fabian Reinartz <freinartz@google.com>
 2018-05-28 16:00:36 -04:00
Krasi Georgiev eb6cdbcb5b some test funcitons didn't cleanup after themselves. 2018-02-23 18:45:59 +02:00
Fabian Reinartz a345231a0e Add test for block meta version and comments 2018-02-12 11:46:22 +01:00
Shubheksha Jalan 510dc17066 fix tests, set version when initializing the index writer 2018-01-11 01:03:20 +05:30
Fabian Reinartz da9614fa54 Allow tombstone file to not exist 2017-12-21 12:07:27 +01:00
Fabian Reinartz 54205903f7 Move index and chunk encoders to own packages 2017-12-21 12:02:05 +01:00
Fabian Reinartz adb6be73dd move label read function int labels pkg 2017-12-21 11:55:58 +01:00
Fabian Reinartz 67f0ca8f0e Move index and chunk encoders to own packages 2017-12-21 11:27:54 +01:00
Callum Styan ca4e817372 move test utils to it's own package so we can use it in tests for other packages in tsdb repo 2017-12-06 17:06:14 -08:00
Goutham Veeramachaneni f35e9c7fec
Merge branch 'master' into fix-dels 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-11-30 13:30:13 +05:30
Goutham Veeramachaneni 1fc94a02d1
Don't retry failed compactions. 
Fixes prometheus/prometheus#3487

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-11-23 15:16:20 +05:30
Goutham Veeramachaneni 44e9ae38b5
Incorporate PR feedback. 
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-26 21:26:31 +05:30
Goutham Veeramachaneni 34a86af3c6
Move tombstones to their own thing. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-17 08:36:56 +05:30
Goutham Veeramachaneni 3de55171d3
Add tests for tombstones and deletedIterator 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-16 12:43:33 +05:30