Commit graph

81 commits

Author SHA1 Message Date
Matthieu MOREL e2ede285a2
refactor: move from io/ioutil to io and os packages (#10528)
* refactor: move from io/ioutil to io and os packages
* use fs.DirEntry instead of os.FileInfo after os.ReadDir

Signed-off-by: MOREL Matthieu <matthieu.morel@cnp.fr>
2022-04-27 11:24:36 +02:00
Peter Štibraný cd18da3605
Fix OpenMetrics parser to sort uppercase labels correctly. (#10510)
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2022-03-31 20:18:53 +02:00
Nick Pillitteri 53ac9d6d66
Compare equality of label.Label structs directly (#10427)
* labels.Equal benchmark for equal, not equal, and differing lengths

Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>

* Compare equality of label.Label structs directly

Compare the structs using `==` instead of the name and value
of each label. This is functionally equivalent and about ~10%
faster in my testing.

Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>

* Use longer more realistic names and values in benchmark

Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>
2022-03-15 00:30:04 +01:00
Levi Harrison a9e2ffb717
Specify type in metadata parser errors (#10269) 2022-03-08 08:04:11 -05:00
Łukasz Mierzwa da23c4649a
Enable misspell check in golangci-lint (#10393)
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2022-03-03 18:11:19 +01:00
Matthias Loibl afdd1357e0
Merge pull request #10267 from roidelapluie/contenttype
Improve content-type error handling
2022-02-14 11:57:09 +01:00
Julien Pivotto 9a2e93228e
Switch to grafana/regexp everywhere (#10268)
Let's have a consistent library for regexp.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2022-02-13 00:58:27 +01:00
Bryan Boreham 579331446a
Allow downstream projects to use faster regexp (#10251)
* Add benchmark for FastRegexMatcher

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Use modified regexp package with optimisations

See https://github.com/grafana/regexp/tree/speedup#readme

Includes the following changes proposed upstream:
* [regexp: allow patterns with no alternates to be one-pass](https://go-review.googlesource.com/c/go/+/353711)
* [regexp: speed up onepass prefix check](https://go-review.googlesource.com/c/go/+/354909)
* [regexp: handle prefix string with fold-case](https://go-review.googlesource.com/c/go/+/358756)
* [regexp: avoid copying each instruction executed](https://go-review.googlesource.com/c/go/+/355789)
* [regexp: allow prefix string anchored at beginning](https://go-review.googlesource.com/c/go/+/377294)

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Use regexp code identical to upstream Go

Change `grafana/regexp` import to use `main` branch.

This means Prometheus is not using the proposed optimisations, but
downstream users of Prometheus code are able to `replace` the library
with the `speedup` branch which does.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-02-08 11:03:20 +01:00
Julien Pivotto f695df843f Improve content-type error handling
- Call err everywhere
- Change log message to underscore-separated field

Followup on #10186

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2022-02-08 11:02:51 +01:00
Matheus Pimenta 8d8ce641a4
error for invalid media type should not be completely swallowed (#10186)
* error for invalid media type should not be completely swallowed

Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2022-02-08 10:57:56 +01:00
ozairasim 52159840eb
Expose Error function for WrappedError (#9662) 2021-11-12 17:38:59 -05:00
beorn7 c954cd9d1d Move packages out of deprecated pkg directory
This creates a new `model` directory and moves all data-model related
packages over there:
  exemplar labels relabel rulefmt textparse timestamp value

All the others are more or less utilities and have been moved to `util`:
  gate logging modetimevfs pool runtime

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-09 08:03:10 +01:00
Bjoern Rabenstein 006b5517e2 Simplify makefiles.
This removes the dependancy on C leveldb and snappy.
It also takes care of fewer dependencies as they would
anyway not work on any non-Debian, non-Brew system.

Change-Id: Ia70dce1ba8a816a003587927e0b3a3f8ad2fd28c
2014-11-25 17:10:39 +01:00
Julius Volz ae30453214 Add label names -> label values index.
Change-Id: Ie39b4044558afc4d1aa937de7dcf8df61f821fb4
2014-03-28 15:16:37 +01:00
Julius Volz 1eee448bc1 Store samples in custom binary encoding.
This has been shown to provide immense decoding speed benefits.

See also:

https://groups.google.com/forum/#!topic/prometheus-developers/FeGl_qzGrYs

Change-Id: I7d45b4650e44ddecaa91dad9d7fdb3cd0b9f15fe
2014-03-09 22:31:38 +01:00
Julius Volz a310afd28f Update "build" -> ".build" in remaining Makefiles. 2013-06-26 16:09:37 +02:00
Matt T. Proud 30b1cf80b5 WIP - Snapshot of Moving to Client Model. 2013-06-25 15:52:42 +02:00
Julius Volz 0226d1ac7a Implement alerts dashboard and expression console links. 2013-06-13 22:35:40 +02:00
Matt T. Proud f895acb9ef Trailing build system cleanups.
The .gitignore files had problems, and the build steps could be
simplified.
2013-06-13 15:38:03 +02:00
Julius Volz fc97e688c6 Improve printing of rules and expressions. 2013-06-11 11:39:31 +02:00
Julius Volz 74cb676537 Implement Stringer interface for rules and all their children. 2013-06-07 15:54:32 +02:00
Julius Volz dcfd09c801 Prepend "exporter_" to labels that already exist in exported metrics.
If the metrics exported by a process already contain any of a target's
base labels (such as "job" or "instance", but also any manually assigned
target-group label), don't overwrite that label, but instead add a new
label consisting of the original label name prepended with "exporter_".
This is to accomodate intermediate exporter jobs, which might indicate
e.g. the jobs and instances for which they are exporting data.
2013-06-02 22:48:46 +02:00
Julius Volz 5b105c77fc Repointerize fingerprints. 2013-05-21 14:28:14 +02:00
Julius Volz 259a0ea467 Optimize fingerprint comparison time. 2013-05-21 14:00:52 +02:00
Matt T. Proud 8f4c7ece92 Destroy naked returns in half of corpus.
The use of naked return values is frowned upon.  This is the first
of two bulk updates to remove them.
2013-05-16 10:53:25 +03:00
Matt T. Proud 244a4a9cdb Update to go1.1.
This commit updates the documentation, Makefiles, formatting, and
code semantics to support the 1.1. runtime, which includes ...

1. ``make advice``,

2. ``make format``, and

3. ``go fix`` on various targets.
2013-05-14 12:39:08 +02:00
Matt T. Proud 161c8fbf9b Include deletion processor for long-tail values.
This commit extracts the model.Values truncation behavior into the actual
tiered storage, which uses it and behaves in a peculiar way—notably the
retention of previous elements if the chunk were to ever go empty.  This is
done to enable interpolation between sparse sample values in the evaluation
cycle.  Nothing necessarily new here—just an extraction.

Now, the model.Values TruncateBefore functionality would do what a user
would expect without any surprises, which is required for the
DeletionProcessor, which may decide to split a large chunk in two if it
determines that the chunk contains the cut-off time.
2013-05-10 12:19:12 +02:00
Matt T. Proud 38d839d810 Include generated Protocol Buffers artifacts.
This commit reduces the general compile time dependencies to omit
the Protocol Buffer compiler and the Go Protocol Buffer generator
tool.  The build steps to furnish them still remain, but they can
optionally be called if data.proto or config.proto are under work.
2013-05-08 21:16:39 +02:00
Matt T. Proud 3b9b1c6ab4 Define dependencies for web. stack concretely.
This commit destroys the use of AppState, which makes passing
concrete state along to various serving components onerous.
2013-05-06 11:13:12 +02:00
Matt T. Proud 6fac20c8af Harden the tests against OOMs.
This commit employs explicit memory freeing for the in-memory storage
arenas.  Secondarily, we take advantage of smaller channel buffer sizes
in the test.
2013-04-29 11:46:01 +02:00
Bernerd Schaefer b04cd28862 Merge pull request #192 from prometheus/feature/negotiate-telemetry-schema-through-mime-type
Use Content-Type data for telemetry versioning
2013-04-29 01:30:37 -07:00
Matt T. Proud b3e34c6658 Implement batch database sample curator.
This commit introduces to Prometheus a batch database sample curator,
which corroborates the high watermarks for sample series against the
curation watermark table to see whether a curator of a given type
needs to be run.

The curator is an abstract executor, which runs various curation
strategies across the database.  It remarks the progress for each
type of curation processor that runs for a given sample series.

A curation procesor is responsible for effectuating the underlying
batch changes that are request.  In this commit, we introduce the
CompactionProcessor, which takes several bits of runtime metadata and
combine sparse sample entries in the database together to form larger
groups.  For instance, for a given series it would be possible to
have the curator effectuate the following grouping:

- Samples Older than Two Weeks: Grouped into Bunches of 10000
- Samples Older than One Week: Grouped into Bunches of 1000
- Samples Older than One Day: Grouped into Bunches of 100
- Samples Older than One Hour: Grouped into Bunches of 10

The benefits hereof of such a compaction are 1. a smaller search
space in the database keyspace, 2. better employment of compression
for repetious values, and 3. reduced seek times.
2013-04-27 17:38:18 +02:00
juliusv 348b527f89 Merge pull request #194 from prometheus/julius-alert-tracking
Track alerts over time and write out alert timeseries.
2013-04-26 05:40:36 -07:00
Julius Volz 2202cd71c9 Track alerts over time and write out alert timeseries. 2013-04-26 14:35:21 +02:00
Bernerd Schaefer dfd5c9ce28 Refactor processor for 0.0.2 schema
Primary changes:

* Strictly typed unmarshalling of metric values
* Schema types are contained by the processor (no "type entity002")

Minor changes:

* Added ProcessorFunc type for expressing processors as simple
  functions.
* Added non-destructive `Merge` method to `model.LabelSet`
2013-04-26 11:52:26 +02:00
Julius Volz d4ff85db5a Add instance label to health (up) timeseries. 2013-04-24 21:50:49 +02:00
Matt T. Proud 9e02c2393a Include generated Protocol Buffer descriptor.
The Protocol Buffer compiler supports generating a machine-readable
descriptor file encoded as a provided Protocol Buffer message type,
which can be used to decode messages that have been encoded with it
after-the-fact.  The generated descriptor also bundles in dependent
message types.

We can use this to perform forensics on old Prometheus clients, if
necessary.
2013-04-24 16:59:40 +02:00
juliusv af7ddc36e2 Merge pull request #176 from prometheus/optimization/view-materialization/slice-chunking
Truncate irrelevant chunk values.
2013-04-24 05:19:54 -07:00
Julius Volz 9b8c671ec9 Fixes/cleanups to renderView() samples truncation. 2013-04-24 12:42:58 +02:00
Matt T. Proud e86f4d9dfd Convert time readers to represent time in UTC.
Go's time.Time represents time as UTC in its fundamental data type.
That said, when using ``time.Unix(...)``, it sets the zone for the
time representation to the local.  Unfortunately with diagnosis and
our tests, it is a PITA to jump between various zones, even though
the serialized version remains the same.

To keep things easy, all places where times are generated or read
are converted into UTC.  These conversions are cheap, for
``Time.In`` merely changes a pointer reference in the struct,
nothing more.  This enables me to diagnose test failures with fixture
data very easily.
2013-04-24 12:19:41 +02:00
Matt T. Proud 05504d3642 WIP - Truncate irrelevant chunk values.
This does not work with the view tests.
2013-04-24 11:07:22 +02:00
Matt T. Proud b1a8e51b07 Extract dto.SampleValueSeries into model.Values. 2013-04-22 13:31:11 +02:00
Matt T. Proud db4ffbb262 Wrap dto.SampleKey with business logic type.
The curator work can be done easier if dto.SampleKey is no longer
directly accessed but rather has a higher level type around it that
captures a certain modicum of business logic.  This doesn't look
terribly interesting today, but it will get more so.
2013-04-21 20:38:39 +02:00
Matt T. Proud f9e99bd08a Refresh SampleValue to 64-bit floating point.
We always knew that this needed to be fixed.
2013-04-21 20:31:50 +02:00
Julius Volz 99dcbe0f94 Integrate memory and disk layers in view rendering. 2013-04-19 16:01:27 +02:00
juliusv 7c544c91c6 Merge pull request #151 from prometheus/julius-synth-vars
Record scrape health timeseries per target.
2013-04-17 05:48:26 -07:00
Matt T. Proud 32c0a939b0 New self-sustaining, hermetic build system.
This should help us produce self-contained artifacts for users as
well as lower the hurdles for first-time contributors.
2013-04-17 09:34:24 +02:00
Julius Volz 8c9e9632a8 Record scrape health timeseries per target. 2013-04-16 19:01:26 +02:00
Julius Volz a0d311c9e6 Constantize job name label. 2013-04-15 11:47:54 +02:00
Julius Volz fd554e8b89 Make LevelDB sample keys fixed-length to ensure correct sort-order. 2013-04-11 18:17:44 +02:00