mudhorn/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2024-11-10 07:34:04 -08:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
1075 commits 245 branches 436 tags 650 MiB
Commit graph

1075 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ganesh Vernekar b1cd829030
Reuse Chunk Iterator (#642) 
* Reset method for chunkenc.Iterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Reset method only for XORIterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Use Reset(...) in querier.go

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Reuse deletedIterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Another way of reusing chunk iterators

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Unexport xorIterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix memSeries.iterator(...)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add some comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-07-09 15:19:34 +05:30
Ganesh Vernekar 8dfa537843
Re-use 'keys' in ReadOffsetTable (#645) 
* Re-use 'keys' in ReadOffsetTable

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add comment

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-07-09 15:16:42 +05:30
Frederic Branczyk d230c67aa1
Merge pull request #649 from prometheus/lr-inject-metrics 
wal: Inject LiveReader metrics rather than registry
 2019-07-03 14:58:22 +02:00
Frederic Branczyk 078895bf56
wal: Inject LiveReader metrics rather than registry 
LiveReaders are instantiated `number of remote write queues * segments`
times, which would cause double registration of the metrics. Instead
this should be orchestrated by the layer above, instantiating the live
reader.

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
 2019-07-03 14:39:25 +02:00
Bartek Płotka 7ef10186b4 Added missing entries to CHANGELOG for 0.9.0 (#648) 
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2019-07-03 14:35:59 +03:00
Bartek Płotka a272e7ec60 Cut TSDB release 0.9.0 (#647) 
Added newline as it is requires for some markdown renderers.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2019-07-03 14:01:53 +03:00
Krasi Georgiev 31f7990d1d
Re-encode chunks that are still being appended to when snapshoti… (#641) 
* re encode all head chunks outside the time range.

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
 2019-07-03 13:47:31 +03:00
Bartek Płotka dc657df591
Merge pull request #644 from codesome/error-wrap 
Check error before wrapping
 2019-07-02 11:53:53 +01:00
Ganesh Vernekar 0dcaf10564
Check error before wrapping 
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-07-02 16:05:27 +05:30
PrometheusBot ee010e9fab Synchronize Makefile.common from prometheus/prometheus (#640) 
Signed-off-by: prombot <prometheus-team@googlegroups.com>
 2019-06-26 12:06:29 +03:00
Krasi Georgiev 8d86e921cf
refactor block size calculation (#637) 
* refactor block size calculation
The block size is kept in memory and not in the meta file anymore.
It now includes the size of the meta file itself for a more
correct block size.
It fixes a bug where the size didn't change when calling `block.Delete()`.
Adds a dedicated test to ensure correct block sizes.
This allows opening a db in a read only mode as it doesn't write to the meta file anymore.

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
 2019-06-24 18:42:29 +03:00
Bartek Płotka b5f9f9f0b4
Merge pull request #636 from krasi-georgiev/rename-bug 
fileutil.Replace - remove destination only when a directory.
 2019-06-24 13:17:41 +01:00
PrometheusBot d501ae4f6e makefile: update Makefile.common with newer version (#638) 
Signed-off-by: prombot <prometheus-team@googlegroups.com>
 2019-06-24 11:52:42 +03:00
Krasi Georgiev 251c3a4d64 comment 
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
 2019-06-24 11:51:49 +03:00
Krasi Georgiev fe3d374586 Merge branch 'master' into rename-bug 
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
 2019-06-24 11:50:41 +03:00
Chris Marchbanks 92d15169fc Print label names with highest cumulative label value length during tsdb analyze (#626) 
Signed-off-by: Ryan Scott <rscott@splunk.com>
Co-authored-by: Chris Marchbanks <csmarchbanks@gmail.com>
 2019-06-21 11:07:50 +03:00
Chris Marchbanks b40cc43958 Provide option to compress WAL records (#609) 
In running Prometheus instances, compressing the records was shown to
reduce disk usage by half while incurring a negligible CPU cost.

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
 2019-06-19 16:46:24 +03:00
Krasi Georgiev ef9b3461c5 fixed the err handling and better comment 
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
 2019-06-19 14:27:19 +03:00
Krasi Georgiev 5d27fc48a8 fileutil.Replace - remove destination only when a directory. 
In cases where a rename fails the fileutil.Replace would delete the
source files/folder.

There is no easy way to make directory renaming atomic, but for files
os.Rename is atomic and replaced the destination file so there is no
need to remove the destination file explicitly.

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
 2019-06-19 12:10:51 +03:00
Björn Rabenstein c2c921af75 Update prometheus/client_golang to v1.0.0 (#635) 
Signed-off-by: beorn7 <beorn@grafana.com>
 2019-06-17 20:54:10 +03:00
Krasi Georgiev 69740485c1
move the wal repair logic in db.Open (#633) 
* move the wal repair logic in db.Open

This is to allow opening a wal in a read oly mode without triggering a
repair.

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
 2019-06-14 17:39:22 +02:00
Björn Rabenstein 4892da59eb
Merge pull request #628 from prometheus/beorn7/client_golang 
Make objectives of Summaries explicit
 2019-06-14 15:19:59 +02:00
naivewong b05ca5c36e fix two benchmarks (#631) 
* fix two benchmarks

Signed-off-by: naivewong <867245430@qq.com>
 2019-06-14 14:30:49 +02:00
beorn7 90a7612df3 Make objectives of Summaries explicit 
With the next release of client_golang, Summaries will not have
objectives by default.

As it turns out, for prometheus_tsdb_head_gc_duration_seconds and
prometheus_tsdb_wal_truncate_duration_seconds, the objective-less
default makes more sense then the current default.

To make sure we do the right thing before and after the upcoming
release of client_golang, I have set the objectives explicitly
wherever that was not the case so far:

- prometheus_tsdb_head_gc_duration_seconds and
  prometheus_tsdb_wal_truncate_duration_seconds now have no objectives
  explicitly.
- prometheus_tsdb_wal_fsync_duration_seconds now explicitly uses the
  previous default objectives.

Signed-off-by: beorn7 <beorn@grafana.com>
 2019-06-14 14:17:24 +02:00
PrometheusBot cb4a085709 Synchronize Makefile.common from prometheus/prometheus (#632) 
Signed-off-by: prombot <prometheus-team@googlegroups.com>
 2019-06-14 10:53:17 +02:00
Krasi Georgiev 3549c097b4
add some comments for the different WAL repair tests (#630) 
The WAL repair tests look similar so added some extra comments to
clarify their purpose.

Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
 2019-06-12 16:10:37 +02:00
naivewong 6ab483071a Improvement on postings intersection (#616) 
* improvement on postings intersection

Signed-off-by: naivewong <867245430@qq.com>
 2019-06-11 10:14:25 +02:00
Krasi Georgiev e809cb477d
Add e2e benchmarks for head and block Queries (#467) 
* Add e2e had and block query benchmarks

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-06-07 15:41:44 +02:00
Brian Brazil be4edbe174
Start a new WAL segement on head truncation. (#605) 
This reduces disk space usage to not be a minimum of 3 128MB files
in small setups. This will possibly also help debug wal data issues,
by making things a bit more deterministic.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2019-06-07 11:35:02 +01:00
Krasi Georgiev 48c4dc63ea
github template for PRs (#624) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-06-07 10:43:04 +02:00
PrometheusBot 4b9970053f makefile: update Makefile.common with newer version (#625) 
Signed-off-by: prombot <prometheus-team@googlegroups.com>
 2019-06-07 10:03:51 +02:00
Brian Brazil 149c5dc73a
Handle multiple refs for the same series when WAL reading. (#623) 
This can happen if a given series is created/truncated/recreated.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2019-06-06 14:28:54 +01:00
Frederic Branczyk 731ed22ac8
Merge pull request #622 from prometheus/live-reader-metric-suffix 
wal: Add `_total` suffix as metric is a counter
 2019-06-05 10:02:23 +02:00
Frederic Branczyk e708ba1d14
wal: Add _total suffix as metric is a counter 
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
 2019-06-05 09:24:25 +02:00
mrasu 3db36f021b Fix make bench (#610) 
Remove  flag -log.level
Signed-off-by: mrasu <m.rasu.hitsuji@gmail.com>
 2019-06-03 22:37:24 +02:00
Frederic Branczyk b800a89ae3
Merge pull request #619 from prometheus/remove-promauto 
wal: Replace promauto with injected registry
 2019-06-03 20:07:15 +02:00
Frederic Branczyk abd15845e2
wal: Replace promauto with injected registry 
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
 2019-06-03 12:29:52 +02:00
Krasi Georgiev 882162d5b9
fix the "failed compaction" metric. (#613) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-05-30 13:57:28 +02:00
naivewong 13c80a5979 Optimize queries using regex matchers for set lookups (#602) 
* Original version of the set optimization

Signed-off-by: naivewong <867245430@qq.com>

* simple set matcher

Signed-off-by: naivewong <867245430@qq.com>

* simple set matcher

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* add benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update benchmark

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>

* use genSeries from #467

Signed-off-by: naivewong <867245430@qq.com>

* update

Signed-off-by: naivewong <867245430@qq.com>
 2019-05-27 16:54:46 +05:30
Callum Styan 562e93e8e6 Always create a new clean segment when starting the WAL. (#608) 
* Always create a new clean segment when starting the WAL.
* Ensure we flush the last page after repairing and before recreating the
new segment in Repair.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
 2019-05-24 19:33:27 +01:00
Krasi Georgiev d48a5e2d5c
Release 0.8.0 (#607) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-05-17 13:39:03 +03:00
Callum Styan bce663e1d9 Export the current segment index as a metic. (#601) 
* Export the current segment index as a metic.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
 2019-05-17 11:47:42 +03:00
Krasi Georgiev 96a87845cc
fix wal panic when page flush fails. (#582) 
* fix wal panic when page flush fails.

New records should be added to the page only when the last flush
succeeded. Otherwise the page would be full and panics when trying to
add a new record.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-05-16 16:40:43 +03:00
Brian Brazil 30d0ea59d7 Don't crash on an unknown tombstone ref. (#604) 
Fixes https://github.com/prometheus/prometheus/issues/5562

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2019-05-16 16:36:44 +03:00
PrometheusBot 63d9c2ab18 makefile: update Makefile.common with newer version (#603) 
Signed-off-by: prombot <prometheus-team@googlegroups.com>
 2019-05-15 10:15:39 +03:00
Brian Brazil 89a90fe96c
Simplify mergedPostings.Seek (#595) 
The current implementation leads to very slow behaviour when there's
many lists, this no worse than n log k, where k is the number of posting
lists.

Adjust benchmark to catch this.

Remove flattening of without lists, not needed anymore.

Benchmark versus 0.4.0 (used in Prometheus 2.7):
```
benchmark                                                            old ns/op      new ns/op      delta
BenchmarkHeadPostingForMatchers/n="1"-8                              189907976      188863880      -0.55%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-8                      113950106      110791414      -2.77%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-8                      104965646      102388760      -2.45%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-8                     138743592      104424250      -24.74%
BenchmarkHeadPostingForMatchers/i=~".*"-8                            5279594954     5206096267     -1.39%
BenchmarkHeadPostingForMatchers/i=~".+"-8                            8004610589     6184527719     -22.74%
BenchmarkHeadPostingForMatchers/i=~""-8                              2476042646     1003920432     -59.45%
BenchmarkHeadPostingForMatchers/i!=""-8                              7178244655     6059725323     -15.58%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-8              199342649      166642946      -16.40%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-8       215774683      167515095      -22.37%
BenchmarkHeadPostingForMatchers/n="1",i!=""-8                        2214714769     392943663      -82.26%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-8                2148727410     322289262      -85.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-8              2170658009     338458171      -84.41%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-8             235720135      70597905       -70.05%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-8       2190570590     343034307      -84.34%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-8     2373784439     387297908      -83.68%

benchmark                                                            old allocs     new allocs     delta
BenchmarkHeadPostingForMatchers/n="1"-8                              33             33             +0.00%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-8                      33             33             +0.00%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-8                      33             33             +0.00%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-8                     41             39             -4.88%
BenchmarkHeadPostingForMatchers/i=~".*"-8                            56             56             +0.00%
BenchmarkHeadPostingForMatchers/i=~".+"-8                            251577         100115         -60.21%
BenchmarkHeadPostingForMatchers/i=~""-8                              251123         100077         -60.15%
BenchmarkHeadPostingForMatchers/i!=""-8                              251525         100112         -60.20%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-8              42             39             -7.14%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-8       52             42             -19.23%
BenchmarkHeadPostingForMatchers/n="1",i!=""-8                        251069         100101         -60.13%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-8                251473         100101         -60.19%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-8              250914         100102         -60.11%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-8             30038          11181          -62.78%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-8       250813         100105         -60.09%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-8     281503         111260         -60.48%

benchmark                                                            old bytes     new bytes     delta
BenchmarkHeadPostingForMatchers/n="1"-8                              10887600      10887600      +0.00%
BenchmarkHeadPostingForMatchers/n="1",j="foo"-8                      5456416       5456416       +0.00%
BenchmarkHeadPostingForMatchers/j="foo",n="1"-8                      5456416       5456416       +0.00%
BenchmarkHeadPostingForMatchers/n="1",j!="foo"-8                     5456640       5456544       -0.00%
BenchmarkHeadPostingForMatchers/i=~".*"-8                            258254504     258254472     -0.00%
BenchmarkHeadPostingForMatchers/i=~".+"-8                            520126192     281554792     -45.87%
BenchmarkHeadPostingForMatchers/i=~""-8                              263446640     24908456      -90.55%
BenchmarkHeadPostingForMatchers/i!=""-8                              520121144     281553664     -45.87%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",j="foo"-8              7062448       7062272       -0.00%
BenchmarkHeadPostingForMatchers/n="1",i=~".*",i!="2",j="foo"-8       7063473       7062384       -0.02%
BenchmarkHeadPostingForMatchers/n="1",i!=""-8                        274325656     35793776      -86.95%
BenchmarkHeadPostingForMatchers/n="1",i!="",j="foo"-8                268926824     30362624      -88.71%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",j="foo"-8              268882992     30363000      -88.71%
BenchmarkHeadPostingForMatchers/n="1",i=~"1.+",j="foo"-8             33193401      4269304       -87.14%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!="2",j="foo"-8       268875024     30363096      -88.71%
BenchmarkHeadPostingForMatchers/n="1",i=~".+",i!~"2.*",j="foo"-8     300589656     33099784      -88.99%
```

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2019-05-13 10:51:07 +01:00
Brian Brazil 6ac81cc7a9
Correctly handle empty labels. (#594) 
Currently a time series with empty labels is not treated the same
as one with missing labels. Currently this can only come from
ALERTS&ALERT_FOR_STATE so it's unlikely anyone has actually hit it.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
 2019-05-07 11:00:16 +01:00
Goutham Veeramachaneni 2ae028114c
Add FromData function (again)! (#600) 
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
 2019-05-06 19:17:26 +05:30
PrometheusBot 4da65c46c9 makefile: update Makefile.common with newer version (#597) 
Signed-off-by: prombot <prometheus-team@googlegroups.com>
 2019-05-05 19:17:35 +03:00
zhulongcheng 3ccab17f5d docs: add label offset table and postings offset table (#593) 
Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
 2019-05-02 23:23:58 +03:00