Commit graph

942 commits

Author SHA1 Message Date
Marco Pracucci 7ad111b27e
Merge remote-tracking branch 'remotes/prometheus/main' into update-upstream 2023-07-06 15:13:54 +02:00
Marc Tudurí 4851ced266
tsdb: Support native histograms in snapshot on shutdown (#12258)
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
2023-07-05 11:44:13 +02:00
Bryan Boreham 013fdb9460 Update goleak for change in glog
Evidently the internal type changed.
2023-07-04 15:01:02 +00:00
Marco Pracucci 34e7d9d997 Fix race condition in ChunkDiskMapper.Truncate() (#12500)
* Fix race condition in ChunkDiskMapper.Truncate()

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Added unit test

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Update tsdb/chunks/head_chunks.go

Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-07-04 15:01:02 +00:00
Justin Lei 92668d9bf7 Also pass chunkOpts into appendPreprocessor
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-07-04 15:01:01 +00:00
Justin Lei 7622d5ac5f Group args to append to memSeries in chunkOpts
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-07-04 15:01:01 +00:00
Nidhey Nitin Indurkar e970f70ced Feat: Get block by id directly on promtool analyze & get latest block if ID not provided (#12031)
* feat: analyze latest block or block by ID in CLI (promtool)

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>

* address remarks

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

* address latest review comments

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

---------

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>
Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>
2023-07-04 13:39:01 +00:00
Alan Protasio e18ee99544 Opmizing Group Regex (#12375)
Signed-off-by: Alan Protasio <alanprot@gmail.com>
2023-07-04 13:39:01 +00:00
Matthieu MOREL 584966f1c4 ci(lint): enable predeclared linter
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-07-04 13:38:31 +00:00
Justin Lei d0fea47a9c Remove samplesPerChunk from memSeries (#12390)
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-07-04 13:38:31 +00:00
Julien Pivotto 9ff1f24efa
Merge pull request #12505 from pracucci/fix-infinite-loop-in-index-writer
Fix infinite loop in index Writer when a series contains duplicated label names
2023-07-04 13:08:36 +02:00
Patrick Oyarzun 68e5937474
Apply relevant label matchers in LabelValues before fetching extra postings (#12274)
* Apply matchers when fetching label values

Signed-off-by: Patrick Oyarzun <patrick.oyarzun@grafana.com>

* Avoid extra copying of label values

Signed-off-by: Patrick Oyarzun <patrick.oyarzun@grafana.com>

---------

Signed-off-by: Patrick Oyarzun <patrick.oyarzun@grafana.com>
2023-07-04 10:37:58 +01:00
Marco Pracucci 35069910f5
Fix infinite loop in index Writer when a series contains duplicated label names
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-07-01 17:38:08 +02:00
Marco Pracucci 031d22df9e
Fix race condition in ChunkDiskMapper.Truncate() (#12500)
* Fix race condition in ChunkDiskMapper.Truncate()

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Added unit test

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Update tsdb/chunks/head_chunks.go

Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-06-30 18:29:59 +05:30
Bartlomiej Plotka 4062f12573
Merge pull request #12396 from leizor/leizor/chunk-opts
Group args to append to memSeries in chunkOpts
2023-06-27 13:08:21 +02:00
Nidhey Nitin Indurkar a8772a4178
Feat: Get block by id directly on promtool analyze & get latest block if ID not provided (#12031)
* feat: analyze latest block or block by ID in CLI (promtool)

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>

* address remarks

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

* address latest review comments

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

---------

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>
Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>
2023-06-01 17:13:09 +05:30
Alan Protasio 73078bf738
Opmizing Group Regex (#12375)
Signed-off-by: Alan Protasio <alanprot@gmail.com>
2023-05-30 13:49:22 +02:00
Julien Pivotto 6f97641a51
Merge pull request #12380 from mmorel-35/patch-2
ci(lint): enable predeclared linter
2023-05-28 14:43:29 +02:00
Justin Lei e73d8b2084 Also pass chunkOpts into appendPreprocessor
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-05-25 13:37:18 -07:00
Justin Lei 4c4454e4c9 Group args to append to memSeries in chunkOpts
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-05-25 13:12:46 -07:00
Justin Lei 89af351730
Remove samplesPerChunk from memSeries (#12390)
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-05-25 11:18:41 +02:00
Jeanette Tan dd172440e5 Merge remote-tracking branch 'mine/fix-default-samples-per-chunk' into zenador/sync-upstream-22-may-2023 2023-05-24 19:32:04 +08:00
zenador 37e5249e33
Use DefaultSamplesPerChunk in tsdb (#12387)
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-05-24 13:00:21 +02:00
Jeanette Tan 02e113d03f Use DefaultSamplesPerChunk in tsdb
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-05-24 16:44:53 +08:00
Jeanette Tan 9f1baaaa38 Fix errors from merge 2023-05-23 01:10:09 +08:00
Jeanette Tan 1be0816b46 Merge remote-tracking branch 'upstream/main' 2023-05-23 00:20:36 +08:00
Baskar Shanmugam 905a0bd63a
Added 'limit' query parameter support to /api/v1/status/tsdb endpoint (#12336)
* Added 'topN' query parameter support to /api/v1/status/tsdb endpoint

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

* Updated query parameter for tsdb status to 'limit'

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

* Corrected Stats() parameter name from topN to limit

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

* Fixed p.Stats CI failure

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

---------

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>
2023-05-22 14:37:07 +02:00
Alan Protasio 8c5d4b4add
Opmize MatchNotEqual (#12377)
Signed-off-by: Alan Protasio <alanprot@gmail.com>
2023-05-21 10:41:30 +02:00
Matthieu MOREL c8e7f95a3c ci(lint): enable predeclared linter
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-05-21 07:33:54 +00:00
George Krajcsovits 92d6980360
Fix populateWithDelChunkSeriesIterator and gauge histograms (#12330)
Use AppendableGauge to detect corrupt chunk with gauge histograms.
Detect if first sample is a gauge but the chunk is not set up to contain
gauge histograms.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2023-05-19 10:24:06 +02:00
Baskar Shanmugam f731a90a7f
Fix LabelValueStats in posting stats (#12342)
Problem:
LabelValueStats - This will provide a list of the label names and memory used in bytes.
It is calculated by adding the length of all values for a given label name.
But internally Prometheus stores the name and the value independently for each series.

Solution:
MemPostings struct maintains the values to seriesRef map which is used
to get the number of series which contains the label values.
Using that LabelValueStats is calculated as: seriesCnt * len(value
name)

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>
2023-05-19 09:36:30 +02:00
Xiaochao Dong 80b7f73d26
Copy tombstone intervals to avoid race (#12245)
Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>
2023-05-17 15:15:12 +02:00
Björn Rabenstein 30e263cf96
Merge pull request #12357 from krajorama/fix-histogram-appendable-emptybucket
fix HistogramAppender.appendable segfault
2023-05-16 20:52:39 +02:00
Callum Styan 0d2108ad79
[tsdb] re-implement WAL watcher to read via a "notification" channel (#11949)
* WIP implement WAL watcher reading via notifications over a channel from
the TSDB code

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Notify via head appenders Commit (finished all WAL logging) rather than
on each WAL Log call

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix misspelled Notify plus add a metric for dropped Write notifications

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Update tests to handle new notification pattern

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* this test maybe needs more time on windows?

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* does this test need more time on windows as well?

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* read timeout is already a time.Duration

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* remove mistakenly commited benchmark data files

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* address some review feedback

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* fix missed changes from previous commit

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix issues from wrapper function

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* try fixing race condition in test by allowing tests to overwrite the
read ticker timeout instead of calling the Notify function

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* fix linting

Signed-off-by: Callum Styan <callumstyan@gmail.com>

---------

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2023-05-15 12:31:49 -07:00
György Krajcsovits c6618729c9 Fix HistogramAppender.Appendable array out of bound error
The code did not handle spans with 0 length properly.

Spans with length zero are now skipped in the comparison.
Span index check not done against length-1, since length is a unit32,
thus subtracting 1 leads to 2^32, not -1.

Fixes and unit tests for both integer and float histograms added.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-05-14 17:38:52 +02:00
Jesus Vazquez 1f1dac2cda
Merge pull request #12351 from alanprot/optimization/MatchNotRegexp
Implementing Regex optimization on the `MatchNotRegexp` matcher type
2023-05-11 11:55:17 +02:00
Alan Protasio c0f1abb574 MatchNotRegexp optimization
Signed-off-by: Alan Protasio <alanprot@gmail.com>
2023-05-10 20:08:38 -07:00
Robert Fratto 9e4e2a4a51
wlog: use filepath for getting checkpoint number
This changes usage of path to be replaced with path/filepath, allowing
for filepath.Base to properly return the base directory on systems where
`/` is not the standard path separator.

This resolves an issue on Windows where intermediate folders containing
a `.` were incorrectly considered to be a part of the checkpoint name.

Related to grafana/agent#3826.

Signed-off-by: Robert Fratto <robertfratto@gmail.com>
2023-05-10 12:38:02 -04:00
Björn Rabenstein 37fe9b89dc
Merge pull request #12055 from leizor/leizor/prometheus/issues/12009
Adjust samplesPerChunk from 120 to 220
2023-05-10 14:45:12 +02:00
Bryan Boreham 0ab9553611
tsdb: drop deleted series from the WAL sooner (#12297)
`head.deleted` holds the WAL segment in use at the time each series was
removed from the head. At the end of `truncateWAL()` we will delete
all segments up to `last`, so we can drop any series that were last seen
in a segment at or before that point.

(same change in Prometheus Agent too)

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-05-01 16:43:15 +01:00
György Krajcsovits 65b8edbed4 Merge remote-tracking branch 'upstream/main' into sync-upstream-28-apr-2023 2023-04-28 18:04:02 +02:00
György Krajcsovits 87c727e957 Fix linter errors in Grafana additions
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-04-28 16:32:42 +02:00
Jeanette Tan a064ec6fc1 Fix lint 2023-04-26 22:22:31 +08:00
Jeanette Tan 0fccba0db9 Merge remote-tracking branch 'upstream/main' 2023-04-26 21:25:21 +08:00
cui fliter 276ca6a883 fix some comments
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-04-25 14:19:16 +08:00
Matthieu MOREL bae9a21200
Merge branch 'main' into linter/nilerr
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-04-19 19:56:39 +02:00
beorn7 5b53aa1108 style: Replace else if cascades with switch
Wiser coders than myself have come to the conclusion that a `switch`
statement is almost always superior to a statement that includes any
`else if`.

The exceptions that I have found in our codebase are just these two:

* The `if else` is followed by an additional statement before the next
  condition (separated by a `;`).
* The whole thing is within a `for` loop and `break` statements are
  used. In this case, using `switch` would require tagging the `for`
  loop, which probably tips the balance.

Why are `switch` statements more readable?

For one, fewer curly braces. But more importantly, the conditions all
have the same alignment, so the whole thing follows the natural flow
of going down a list of conditions. With `else if`, in contrast, all
conditions but the first are "hidden" behind `} else if `, harder to
spot and (for no good reason) presented differently from the first
condition.

I'm sure the aforemention wise coders can list even more reasons.

In any case, I like it so much that I have found myself recommending
it in code reviews. I would like to make it a habit in our code base,
without making it a hard requirement that we would test on the CI. But
for that, there has to be a role model, so this commit eliminates all
`if else` occurrences, unless it is autogenerated code or fits one of
the exceptions above.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-19 17:22:31 +02:00
beorn7 c3c7d44d84 lint: Adjust to the lint warnings raised by current versions of golint-ci
We haven't updated golint-ci in our CI yet, but this commit prepares
for that.

There are a lot of new warnings, and it is mostly because the "revive"
linter got updated. I agree with most of the new warnings, mostly
around not naming unused function parameters (although it is justified
in some cases for documentation purposes – while things like mocks are
a good example where not naming the parameter is clearer).

I'm pretty upset about the "empty block" warning to include `for`
loops. It's such a common pattern to do something in the head of the
`for` loop and then have an empty block. There is still an open issue
about this: https://github.com/mgechev/revive/issues/810 I have
disabled "revive" altogether in files where empty blocks are used
excessively, and I have made the effort to add individual
`// nolint:revive` where empty blocks are used just once or twice.
It's borderline noisy, though, but let's go with it for now.

I should mention that none of the "empty block" warnings for `for`
loop bodies were legitimate.

Signed-off-by: beorn7 <beorn@grafana.com>
2023-04-19 17:10:10 +02:00
Patrick Oyarzun fc3fa8995a
Merge pull request #483 from grafana/logiraptor/apply-matchers-in-labelvalues
Apply matchers when fetching label values
2023-04-18 17:38:07 -05:00
Patrick Oyarzun e52ce3e361
Avoid extra copying of label values 2023-04-18 15:50:05 -05:00