Commit graph

11441 commits

Author SHA1 Message Date
Goutham Veeramachaneni ad4f514e66
Add OTLP Ingestion endpoint (#12571)
* Add OTLP Ingestion endpoint

We copy files from the otel-collector-contrib. See the README in
`storage/remote/otlptranslator/README.md`.

This supersedes: https://github.com/prometheus/prometheus/pull/11965

Signed-off-by: gouthamve <gouthamve@gmail.com>

* Return a 200 OK

It is what the OTEL Golang SDK expect :(

https://github.com/open-telemetry/opentelemetry-go/issues/4363

Signed-off-by: Goutham <gouthamve@gmail.com>

---------

Signed-off-by: gouthamve <gouthamve@gmail.com>
Signed-off-by: Goutham <gouthamve@gmail.com>
2023-07-28 12:35:28 +02:00
Robert Fratto 886945cda7
tsdb/agent: ensure that new series get written to WAL on rollback (#12592)
If a new series is introduced in a storage.Appender instance, that
series should be written to the WAL once the storage.Appender is closed,
even on Rollback.

Previously, new series would only be written to the WAL when calling
Commit. However, because the series is stored in memory regardless,
subsequent calls to Commit may write samples to the WAL which reference
a series ID which that was never written.

Related to #11589. It's likely that this fix also resolves this issue,
but we need more testing from users to see if the problem persists after
this fix; there may be more cases where samples get written to the WAL
in Prometheus Agent mode without the corresponding series record.

Signed-off-by: Robert Fratto <robertfratto@gmail.com>
2023-07-27 09:28:26 -04:00
Ziqi Zhao ed1b307bca
enhance FloatHistogram CopyToSchema method (#12596)
histogram: Improve performance of FloatHistogram.CopyToSchema

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>

---------

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
2023-07-27 13:27:13 +02:00
Björn Rabenstein b1a6d003d1
Merge pull request #12584 from prometheus/beorn7/histogram
histogram: Identify native histograms even without observations
2023-07-27 00:18:17 +02:00
beorn7 c58e20ad0e histogram: Identify native histograms even without observations
Native histograms without observations and with a zero threshold of
zero look the same as classic histograms in the protobuf exposition
format. According to
https://github.com/prometheus/client_golang/issues/1127 , the idea is
to add a no-op span to those histograms to mark them as native
histograms. This commit enables Prometheus to detect that no-op span
and adds a doc comment to the proto spec describing the behavior.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-26 17:42:30 +02:00
George Krajcsovits 6cd2d1621f
Hide histogram chunk append and reset header internals (#12352)
tsdb: Hide histogram chunk append and reset header internals

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2023-07-26 15:08:16 +02:00
Julien Pivotto 03e549cc39
Merge pull request #8747 from shoce/patch-1
Add instructions how to persist prometheus docker container data
2023-07-26 09:02:48 +02:00
Julien Pivotto 657da2eb98
Merge pull request #12595 from prometheus/atan2-drop-metric-name
Drop metric name for "atan2" binary operator
2023-07-24 16:48:10 +02:00
Julius Volz 531567d46e Drop metric name for "atan2" binary operator
The operator changes the meaning of the metric, so the metric name should
be dropped. Technically this would be a breaking change, but it's also very
obviously a bug and not likely that anyone depends on it.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2023-07-24 14:36:02 +02:00
Julien Pivotto 94edd08859
CI: Fix typo: arfefact -> artefacts (#12590)
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-07-22 14:29:34 +01:00
LHHDZ 7d8f9b0978
remote-write receiver: reuse 'ref' to optimize multiple samples for same series (#12580)
reuse 'ref' to optimize multi samples processing efficiency

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2023-07-22 14:24:46 +01:00
Björn Rabenstein 0e12f11d61
Merge pull request #12583 from prometheus/release-2.46
Merge release-2.46 into main
2023-07-20 18:29:44 +02:00
Björn Rabenstein 9228ac16aa
Merge pull request #12582 from krajorama/error-handling-tsdbutil-chunkfromsamples
tsdbutil/ChunkFromSamplesGeneric should not panic
2023-07-20 18:02:16 +02:00
György Krajcsovits d4e355243a tsdbutil/ChunkFromSamplesGeneric should not panic
Add error handling instead.
Prepares for #12352

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-07-20 17:01:34 +02:00
Julien Pivotto 7905594b52
Merge pull request #12557 from prometheus/beorn7/histogram
scrape: Enable ingestion of multiple exemplars per sample
2023-07-20 15:19:28 +02:00
Björn Rabenstein b2567aeacd
Merge pull request #12575 from prometheus/beorn7/histogram2
histogram: Fix bounds of buckets returned by floatBucketIterator
2023-07-20 13:20:51 +02:00
beorn7 9aadd54786 histogram: Fix bounds of buckets returned by floatBucketIterator
The bounds weren't really used so far, so no actual bug in the code so
far. But it's obviously confusing if the bounds returned by a
floatBucketIterator with a target schema different from the original
schema are wrong.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-19 18:19:18 +02:00
Björn Rabenstein 266e27870e
Merge pull request #12577 from prometheus/beorn7/histogram3
histograms: Fix parsing of float histograms without zero bucket
2023-07-19 16:44:27 +02:00
beorn7 071f4bbea4 histograms: Fix parsing float histograms without zero bucket
If a float histogram has a zero bucket with a threshold of zero _and_
an empty zero bucket, it wasn't identified as a native histogram
because the `isNativeHistogram` helper function only looked at integer
buckets.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-19 15:29:11 +02:00
beorn7 2ea8df4734 histogram: Expose #12305
Native histograms without a zero threshold aren't federated properly.

This adds a test to prove the specific failure mode, which is that
histograms with a zero threshold of zero are federated as classic
histograms.

The underlying reason is that the protobuf parser identifies a native
histogram by detecting a zero bucket or by detecting integer buckets.
Therefore, a float histogram with a zero threshold of zero and an
unpopulated zero bucket falls through the cracks (no integer buckets,
no zero bucket).

This commit also addse a test case for the latter.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-19 15:29:11 +02:00
Julien Pivotto f44915e56b
Merge pull request #12578 from roidelapluie/deps-246
Prepare go dependencies for 2.46
2023-07-19 14:35:38 +02:00
Julien Pivotto 5d7bcee12d Prepare go dependencies for 2.46
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-07-19 09:52:38 +02:00
Julien Pivotto 0a48f93111
Merge pull request #10367 from ianwoolf/pr_add_close_for_query_logger
add Close for ActiveQueryTracker to close the file.
2023-07-18 13:53:18 +02:00
Julien Pivotto 1f5934e7be
Merge pull request #10623 from songjiayang/update-index
make sure response error when TOC parse failed
2023-07-18 13:47:27 +02:00
Julien Pivotto 2bdb2e627f
Merge pull request #10914 from wangrzneu/add-endpointslice-label
Add more labels for endpointslice and endpoints role in k8s discovery
2023-07-18 13:35:03 +02:00
Julien Pivotto e0502f00c8
Merge pull request #11285 from ashishkurmi/main
ci: add minimum GitHub token permissions for workflows
2023-07-18 13:22:07 +02:00
Julien Pivotto 782e6f64fb
Merge pull request #11295 from dimitarvdimitrov/dimitar/simplify-evalTimestamp
Simplify rule group's EvalTimestamp formula
2023-07-18 13:21:20 +02:00
Julien Pivotto b7028c1722
Merge pull request #11426 from douglascamata/document-and-test-relabel-memory-reuse
Better document possible relabel input modification
2023-07-18 13:16:54 +02:00
Julien Pivotto 06d31c9cc9
Merge pull request #12569 from hetznercloud/hcloud-go-v2
build(deps): bump github.com/hetznercloud/hcloud-go to v2.0.0
2023-07-17 16:58:15 +02:00
Julien Pivotto a39d2dbb23
Merge pull request #12570 from hetznercloud/pr-signoff
docs: use actual flag for signing off commits
2023-07-17 16:57:22 +02:00
Julian Tölle 5a0dea1d91
docs: use actual flag for signing off commits
Flag is documented as --signoff: https://git-scm.com/docs/git-commit#Documentation/git-commit.txt---signoff

Signed-off-by: Julian Tölle <julian.toelle@hetzner-cloud.de>
2023-07-17 14:30:33 +02:00
Julian Tölle 7aa7965716
build(deps): bump github.com/hetznercloud/hcloud-go to v2.0.0
Version 2 introduced a breaking change in the `id` field of all
resources. They were changed from `int` to `int64` to make sure that all
future numerical IDs are supported on all architectures.

You can learn more about this
[here](https://docs.hetzner.cloud/#deprecation-notices-%E2%9A%A0%EF%B8%8F)

Signed-off-by: Julian Tölle <julian.toelle@hetzner-cloud.de>
2023-07-17 14:25:29 +02:00
Julien Pivotto c204e1b924
Merge pull request #12568 from dswarbrick/linode-int-overflow
Linode SD: cast InstanceSpec values to int64 to avoid overflows
2023-07-17 08:22:25 +02:00
Daniel Swarbrick de89a8c827
Linode SD: cast InstanceSpec values to int64 to avoid overflows
InstanceSpec struct members are untyped integers, so they can overflow
on 32-bit arch when bit-shifted left.

Signed-off-by: Daniel Swarbrick <daniel.swarbrick@gmail.com>
2023-07-16 19:23:44 +02:00
Julien Pivotto 52b1ddc050
Merge pull request #11411 from robskillington/add-promql-cmds-to-promtool
Add PromQL format and label matcher set/delete commands to promtool
2023-07-15 23:19:17 +02:00
Julien Pivotto c572d9d6d9
Merge pull request #11905 from charleskorn/api-response-format-extension-point
Add extension point for returning different content types from API endpoints
2023-07-15 22:49:29 +02:00
Julien Pivotto fd5b01afdc promtool docs: write flags between backtits in help
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-07-15 22:32:59 +02:00
cui fliter 096ceca44f
remove repetitive words (#12556)
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-07-13 15:53:40 +02:00
beorn7 536a487af4 scrape: Refactor names of float samples
Continue to remove confusion that histogram samples are also samples
and histogram values are also values etc. by renaming float values and
float samples using the same schema as for histograms.

Concretely:
- result → resultFloats (corresponding to resultHistograms)
- pendingResult → pendingFloats (corresponding to pendingHistograms)
- rolledbackResult → rolledbackFloats (corresponding to rolledbackHistograms)
- sample → floatSample (corresponding to histogramSample)

This also order the fields in `collectResultAppender` more
consistently.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-13 14:27:51 +02:00
beorn7 0e3f35324b scrape: Enable ingestion of multiple exemplars per sample
This has become a requirement for native histograms, as a single
histogram sample commonly has many buckets, so that providing many
exemplars makes sense.

Since OM text doesn't support native histograms yet, the test had to
be expanded to also support protobuf test cases.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-13 14:16:10 +02:00
Björn Rabenstein cb93a0cbd2
Merge pull request #12554 from prometheus/beorn7/histogram
Histograms: Expose and fix #12552
2023-07-13 11:52:00 +02:00
Julien Pivotto b3b669fd9a Add experimental flag and docs
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-07-12 22:33:49 +02:00
Rob Skillington e1ace8d00e Add PromQL format and label matcher set/delete commands to promtool
Signed-off-by: Rob Skillington <rob@chronosphere.io>
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-07-12 22:33:44 +02:00
beorn7 da047c6857 histograms: Fix bug #12552
The problem was the following:

When trying to parse native histograms and classic histograms in
parallel, the parser would first parse the histogram proto messages as
a native histogram and then parse the same message again, but now as a
classic histogram. Afterwards, it would forget that it was dealing
with a metric family that contains native histograms and would parse
the rest of the metric family as classic histograms only. The fix is
to check again after being done with a classic histogram.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-12 18:47:00 +02:00
beorn7 1c3bd04bea histograms: Modify test to expose bug #12552
Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-12 18:17:35 +02:00
Douglas Camata 47819fef01
Merge branch 'main' of github.com:prometheus/prometheus into document-and-test-relabel-memory-reuse
Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com>
2023-07-12 15:01:29 +02:00
beorn7 162612ea86 histograms: Improve comment
Oversight during review of #12525.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-12 14:52:49 +02:00
Björn Rabenstein 5da638d527
Merge pull request #12525 from fatsheep9146/native-histogram-min-max
* Add function for iterating through all buckets in reverse to find max bucket

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* enhance histogram_quantile to get min/max value

Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>

---------

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
Co-authored-by: Carrie Edwards <edwrdscarrie@gmail.com>
2023-07-12 14:39:57 +02:00
Ziqi Zhao 42d9169ba1 enhance histogram_quantile to get min/max value
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2023-07-12 04:29:54 +08:00
Carrie Edwards f93ac97867 Update querying function docs
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2023-07-11 21:51:20 +08:00