Commit graph

8917 commits

Author SHA1 Message Date
Dieter Plaetinck 6c13375ac8
sparsehistogram recoding upon detection that new buckets have appeared (#9030)
* bucketIterator which returns all valid bucket indices for a []span

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* support for comparing []spans and generating interjections

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* add license header

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* assert order fix

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* handle pathological 0-length span case more gracefully

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* stale todo

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* decode-recode histograms when new buckets appear

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* factor out recoding and also add it to the fallback case

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* make linter happy

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>
2021-07-02 11:50:30 +05:30
beorn7 518b77c59d Fix a few trivial style nits
Signed-off-by: beorn7 <beorn@grafana.com>
2021-07-01 17:11:54 +02:00
Ganesh Vernekar a1087ed37a
Fix scraping of sparse histograms (#9031)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-01 18:19:04 +05:30
Björn Rabenstein dbab4957f7
Merge pull request #9027 from prometheus/beorn7/histogram2
Hacky implementation of protobuf parsing
2021-07-01 12:35:42 +02:00
beorn7 5de2df752f Hacky implementation of protobuf parsing
This "brings back" protobuf parsing, with the only goal to play with
the new sparse histograms.

The Prom-2.x style parser is highly adapted to the structure of the
Prometheus text format (and later OpenMetrics). Some jumping through
hoops is required to feed protobuf into it.

This is not meant to be a model for the final implementation. It
should just enable sparse histogram ingestion at a reasonable
efficiency.

Following known shortcomings and flaws:

- No tests yet.

- Summaries and legacy histograms, i.e. without sparse buckets, are
  ignored.

- Staleness doesn't work (but this could be fixed in the appender, to
  be discussed).

- No tricks have been tried that would be similar to the tricks the
  text parsers do (like direct pointers into the HTTP response
  body). That makes things weird here. Tricky optimizations only make
  sense once the final format is specified, which will almost
  certainly not be the old protobuf format. (Interestingly, I expect
  this implementation to be in fact much more efficient than the
  original protobuf ingestion in Prom-1.x.)

- This is using a proto3 version of metrics.proto (mostly to be
  consistent with the other protobuf uses). However, proto3 sees no
  difference between an unset field. We depend on that to distinguish
  between an unset timestamp and the timestamp 0 (1970-01-01, 00:00:00
  UTC). In this experimental code, we just assume that timestamp is
  never specified and therefore a timestamp of 0 always is interpreted
  as "not set".

Signed-off-by: beorn7 <beorn@grafana.com>
2021-07-01 01:35:11 +02:00
Ganesh Vernekar f4d3af73f0
Query histograms from TSDB and unit test for append+query (#9022)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-30 20:18:13 +05:30
Dieter Plaetinck 4d27816ea5
Sparsehistogram: improve dod encoding, testing, encode chunk metadata (#9015)
* factor out different varbit schemes and include Beorn's "optimum" for buckets

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* use more compact dod encoding scheme for SHS chunk columns

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* remove FB VB and xor dod encoding because we won't use it

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* HistoChunk metadata encoding

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* add SparseHistogram.Copy()

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* histogram test: test appending a few histograms

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* add license headers

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>
2021-06-30 16:15:43 +05:30
Ganesh Vernekar 04ad56d9b8
Append sparse histograms into the Head block (#9013)
* Append sparse histograms into the Head block

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Add AtHistogram() to Iterator interface. Make HistoChunk conform to Chunk interface.

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-29 20:08:46 +05:30
Dieter Plaetinck 58917d1b76
sparsehistogram: integer types and timestamp separation (#9014)
* integer types and timestamp separation

1) unify types to int64. as suggested by beorn. we want to support
   counters going down (resets) even if we plan to create new chunks for
   now, in that case
2) histogram type doesn't know its own timestamp. include it separately
   in appending and iteration

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* correction: count and zeroCount to remain unsigned

to make api more resilient and that's what we use in protobuf anyway

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* temp hack. Ganesh will fix

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>
2021-06-29 19:27:59 +05:30
Dieter Plaetinck fd11a339a7
Sparsehistogram chunk implementation (#9009)
* histogram chunk

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* xorAppender.AppendHistogram non-method

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>

* basic histogram chunk test

Signed-off-by: Dieter Plaetinck <dieter@grafana.com>
2021-06-29 14:07:41 +05:30
Ganesh Vernekar 64bea6999e
HistogramAppender interface for sparse histograms (#9007)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-28 20:30:55 +05:30
Björn Rabenstein df3b674f01
Add metrics.proto to other protobuf code (#8996)
We cannot just use prometheus/client_model directly because we want to
stay consistent with the use of gogo-protobuf. So this converts
metrics.proto to proto3 and edits it lightly so that it fits into
the framework how prometheus/prometheus handles protobuf.

Note that metrics.proto couldn't be merged into the prompb package
because prompb already has an Exemplar type, which is different from
the Exemplar type in metrics.proto. The directory structure seems to
play a role in the protobuf world, so I better kept it.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-06-28 11:16:06 +01:00
Julien Pivotto 3707619900
Merge pull request #8995 from LeviHarrison/reposync-yamllint
Add .yamllint to reposync (and don't run when not installed)
2021-06-25 14:48:35 +02:00
Levi Harrison b7d2f4e189
Only run if yamllint is installed
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-25 08:03:58 -04:00
Levi Harrison b0df5d9568
Added .yamllint to reposync
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-25 07:46:48 -04:00
Ben Kochie ecc5960150
Merge pull request #8991 from roidelapluie/yamllint
Fix yaml indent to make CI happy
2021-06-25 10:04:41 +02:00
Julien Pivotto 17700e5600 Fix yaml indent to make CI happy
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-06-25 00:53:22 +02:00
Levi Harrison d5c3c567d3
Remote Write: Add max samples per metadata send (#8959)
* Added MaxSamplesPerSend

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added tests

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed order of require

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added docs

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* writes -> writesReceived

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Improved send loop

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-24 15:39:50 -07:00
Ben Kochie 7cb55d5732
Merge pull request #8802 from mwasilew2/yaml-linting
Adds yamllinting to Makefile.common
2021-06-24 15:59:35 +02:00
Julien Pivotto d8219f29f4
Merge pull request #8983 from roidelapluie/httpsdtests
HTTP SD: Add tests
2021-06-23 12:29:27 +02:00
Julien Pivotto 1deac8f570 HTTP SD: Add tests
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-06-23 09:40:50 +02:00
Julius Volz 4a5aef0495
Merge release-2.28 back into main (#8975)
* Cut v2.28.0-rc.0 (#8954)

* Cut v2.28.0-rc.0

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Changelog fixup

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Address review comments

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Downgrade some features to enhancements

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Adjust release date to today

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Migrate HTTP SD docs from docs repo (#8972)

See discussion in https://github.com/prometheus/docs/pull/1975

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Cut Prometheus v2.28.0 (#8973)

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-22 00:59:31 +02:00
Julien Pivotto fa6b2897f0
Merge pull request #8956 from LeviHarrison/fix-tsdb-test-flake
CI: Ignore goleak in TSDB test
2021-06-20 23:41:55 +02:00
Julius Volz eda5f8ca5f
Fix bold formatting bug in RELEASE.md (#8962)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-18 15:41:02 +02:00
Julius Volz d98d342f2b
Cut v2.28.0-rc.0 (#8954) (#8963)
* Cut v2.28.0-rc.0

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Changelog fixup

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Address review comments

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Downgrade some features to enhancements

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Adjust release date to today

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-18 15:40:50 +02:00
Julius Volz 8d98750e8f
Simplify float formatting for HTTP SD refresh interval (#8961)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-18 11:49:30 +02:00
Richard Hartmann 6a055f118a
Merge pull request #8960 from prometheus/improve-scrape-timeout-header-formatting
Remove trailing zeros in scrape timeout header
2021-06-18 10:44:01 +02:00
Julius Volz 9d495afd2c Remove trailing zeros in scrape timeout header
See https://twitter.com/AviKivity/status/1405147699557638145 and
https://twitter.com/juliusvolz/status/1405790211670515712

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-18 09:38:12 +02:00
Levi Harrison 437c470c40 Added ignore
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-17 10:04:52 -04:00
Julien Pivotto 60918b8415
Merge pull request #8953 from LeviHarrison/change-godoc-links
Replace `godoc.org` links with `pkg.go.dev` links
2021-06-17 15:36:51 +02:00
Julien Pivotto 9e88c3bb4d
Merge pull request #8952 from roidelapluie/fixstorage
Storage: Create a code block for example command.
2021-06-17 13:24:08 +02:00
Levi Harrison 4a4882d4c7 Replace godoc.org links
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-17 07:18:51 -04:00
Levi Harrison 922f9babb1
Change starting show screen progress bar condition (#8946)
* Show progress bar when max is not 0

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added semicolon

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-17 12:48:38 +02:00
3Xpl0it3r a0bac4b488
add kubeconfig support in discovery module (#8811)
Signed-off-by: 3Xpl0it3r <shouc.wang@hotmail.com>
2021-06-17 12:41:50 +02:00
Julien Pivotto 2c9875d0e7 Storage: Create a code block for example command.
The current doc is broken:
https://prometheus.io/docs/prometheus/2.27/storage/

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-06-17 12:21:45 +02:00
Julien Pivotto b1c179be85
Fix main build (#8948)
Was broken after the merge of #8824

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-06-16 17:18:32 +02:00
Richard Hartmann 202446955a
Merge pull request #8765 from SmedbergM/label-replace-docs
docs :: querying :: functions :: label_replace
2021-06-16 13:01:59 +02:00
Julien Duchesne 8855c2e626
Add prometheus_tsdb_clean_start metric (#8824)
Add cleanup of the lockfile when the db is cleanly closed

The metric describes the status of the lockfile on startup
0: Already existed
1: Did not exist
-1: Disabled

Therefore, if the min value over time of this metric is 0, that means that executions have exited uncleanly
We can then use that metric to have a much lower threshold on the crashlooping alert:

If the metric exists and it has been zero, two restarts is enough to trigger the alarm
If it does not exist (old prom version for example), the current five restarts threshold remains

Signed-off-by: Julien Duchesne <julien.duchesne@grafana.com>

* Change metric name + set unset value to -1

Signed-off-by: Julien Duchesne <julien.duchesne@grafana.com>

* Only check the last value of the clean start alert

Signed-off-by: Julien Duchesne <julien.duchesne@grafana.com>

* Fix test + nit

Signed-off-by: Julien Duchesne <julien.duchesne@grafana.com>
2021-06-16 15:03:02 +05:30
Levi Harrison b59822904d
Add command to docker build instructions (#8944)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-15 23:16:21 +02:00
Julius Volz 00a9d77b1d
Update node dependencies before release (#8937)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-15 23:07:01 +02:00
Julius Volz b12622206d
Improve styling of selected exemplar details (#8945)
This isn't perfect yet, but IMO better than the initial draft.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-15 22:50:01 +02:00
Levi Harrison eb8ca06885
React UI: Add Starting Screen to Individual Pages (#8909)
* Fix/removed forwarding

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added global 'wasReady' and 'wasUnexpected'

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Eslint fixes

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added withStartingIndicator wrapper

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed condition

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Removed unused import

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Moved withStartingIndicator calls to pages index

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed withStartingIndicator tests

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed eslint (maybe?)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Trailing comma

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added prettier ignore

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fix eslint (pt. 2)

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-15 22:37:16 +02:00
smedbergM 034cdb0aa4 Revert change in label_join per @RichiH
Signed-off-by: smedbergM <matthew.smedberg@gmail.com>
2021-06-15 13:55:02 -06:00
smedbergM 918c06feb9 Merge branch 'main' into label-replace-docs 2021-06-15 12:27:09 -06:00
Julien Pivotto ea3728bcac
go.mod: Exclude de-published linode v1.0.0 (#8941)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-06-15 16:06:48 +02:00
Julien Pivotto ba76bceb6b
Merge pull request #8917 from stevekuznetsov/skuznets/silence-backfill
promtool: backfill: allow silencing output
2021-06-14 23:27:18 +02:00
Julius Volz cea9486918
Update Go dependencies before release (#8936)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-14 19:18:28 +02:00
Julius Volz 676acf6a1b
Improve graph info tooltip for series and exemplars (#8929)
* Improve graph info tooltip for series and exemplars

It now shows more clearly what the label sets are about (series vs. exemplar +
associated series) and also shows the metric name for both label sets when it
is present.

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Fix up inconsistent title casing

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Explicitly indicate empty labelsets in hover tooltip

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Adjust tests to new hover detail

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2021-06-14 16:35:21 +02:00
Frederic Branczyk 039b651450
Merge pull request #8916 from Evesy/main
Add class label to kubernetes ingress discovery
2021-06-14 13:40:08 +02:00
Julien Pivotto 9c23d1741a
Merge pull request #8934 from koolwithk/patch-1
Docs - fix wrong spell
2021-06-14 10:22:35 +02:00