Commit graph

958 commits

Author SHA1 Message Date
Jeanette Tan 8035c04624 Merge remote-tracking branch 'upstream/main'
Minor conflicts:
rules/manager.go
tsdb/compact.go
tsdb/db.go
go.mod
2023-07-19 21:40:27 +08:00
Julien Pivotto 1f5934e7be
Merge pull request #10623 from songjiayang/update-index
make sure response error when TOC parse failed
2023-07-18 13:47:27 +02:00
cui fliter 096ceca44f
remove repetitive words (#12556)
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-07-13 15:53:40 +02:00
Julien Pivotto 89e213bc02
Merge pull request #12546 from roidelapluie/removeimport
TSDB: Remove usused import of sort
2023-07-11 15:06:48 +02:00
Justin Lei 32d87282ad
Add Zstandard compression option for wlog (#11666)
Snappy remains as the default compression but there is now a flag to switch 
the compression algorithm.

Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-07-11 14:57:57 +02:00
Julien Pivotto bf5bf1a4b3 TSDB: Remove usused import of sort
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-07-11 14:29:31 +02:00
Julien Pivotto 8c8afec116
Merge pull request #12542 from merrickclay/tsdb-doc-comment
improve incorrect doc comment
2023-07-11 13:10:04 +02:00
Julien Pivotto 0f85e4f41d
Merge pull request #12539 from bboreham/slices-sorts
Replace sort.Slice with faster slices.SortFunc
2023-07-11 13:09:02 +02:00
Merrick Clay 70e41fc5ac improve incorrect doc comment
Signed-off-by: Merrick Clay <merrick.e.clay@gmail.com>
2023-07-10 16:52:00 -06:00
Bryan Boreham ce153e3fff Replace sort.Sort with faster slices.SortFunc
The generic version is more efficient.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-10 09:43:45 +00:00
Charles Korn f3697f5242
Merge pull request #513 from grafana/charleskorn/expose-chunk-count
Add `ChunkCount()` method to `ChunkSeries`
2023-07-07 09:42:45 +10:00
Marco Pracucci 7ad111b27e
Merge remote-tracking branch 'remotes/prometheus/main' into update-upstream 2023-07-06 15:13:54 +02:00
Charles Korn a0634e7408
Rename EstimatedChunkCount to ChunkCount 2023-07-06 15:42:44 +10:00
Charles Korn a9445622ad
Permit returning an error from EstimatedChunkCount() 2023-07-06 15:41:05 +10:00
Marc Tudurí 4851ced266
tsdb: Support native histograms in snapshot on shutdown (#12258)
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
2023-07-05 11:44:13 +02:00
Charles Korn 8b84856ba5
Add EstimatedChunkCount() method to ChunkSeries
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2023-07-05 15:34:07 +10:00
Bryan Boreham 013fdb9460 Update goleak for change in glog
Evidently the internal type changed.
2023-07-04 15:01:02 +00:00
Marco Pracucci 34e7d9d997 Fix race condition in ChunkDiskMapper.Truncate() (#12500)
* Fix race condition in ChunkDiskMapper.Truncate()

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Added unit test

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Update tsdb/chunks/head_chunks.go

Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-07-04 15:01:02 +00:00
Justin Lei 92668d9bf7 Also pass chunkOpts into appendPreprocessor
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-07-04 15:01:01 +00:00
Justin Lei 7622d5ac5f Group args to append to memSeries in chunkOpts
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-07-04 15:01:01 +00:00
Nidhey Nitin Indurkar e970f70ced Feat: Get block by id directly on promtool analyze & get latest block if ID not provided (#12031)
* feat: analyze latest block or block by ID in CLI (promtool)

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>

* address remarks

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

* address latest review comments

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

---------

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>
Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>
2023-07-04 13:39:01 +00:00
Alan Protasio e18ee99544 Opmizing Group Regex (#12375)
Signed-off-by: Alan Protasio <alanprot@gmail.com>
2023-07-04 13:39:01 +00:00
Matthieu MOREL 584966f1c4 ci(lint): enable predeclared linter
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-07-04 13:38:31 +00:00
Justin Lei d0fea47a9c Remove samplesPerChunk from memSeries (#12390)
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-07-04 13:38:31 +00:00
Julien Pivotto 9ff1f24efa
Merge pull request #12505 from pracucci/fix-infinite-loop-in-index-writer
Fix infinite loop in index Writer when a series contains duplicated label names
2023-07-04 13:08:36 +02:00
Patrick Oyarzun 68e5937474
Apply relevant label matchers in LabelValues before fetching extra postings (#12274)
* Apply matchers when fetching label values

Signed-off-by: Patrick Oyarzun <patrick.oyarzun@grafana.com>

* Avoid extra copying of label values

Signed-off-by: Patrick Oyarzun <patrick.oyarzun@grafana.com>

---------

Signed-off-by: Patrick Oyarzun <patrick.oyarzun@grafana.com>
2023-07-04 10:37:58 +01:00
Bryan Boreham 5255bf06ad Replace sort.Slice with faster slices.SortFunc
The generic version is more efficient.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-07-02 22:17:08 +00:00
Marco Pracucci 35069910f5
Fix infinite loop in index Writer when a series contains duplicated label names
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2023-07-01 17:38:08 +02:00
Marco Pracucci 031d22df9e
Fix race condition in ChunkDiskMapper.Truncate() (#12500)
* Fix race condition in ChunkDiskMapper.Truncate()

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Added unit test

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Update tsdb/chunks/head_chunks.go

Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Marco Pracucci <marco@pracucci.com>

---------

Signed-off-by: Marco Pracucci <marco@pracucci.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
2023-06-30 18:29:59 +05:30
Bartlomiej Plotka 4062f12573
Merge pull request #12396 from leizor/leizor/chunk-opts
Group args to append to memSeries in chunkOpts
2023-06-27 13:08:21 +02:00
Nidhey Nitin Indurkar a8772a4178
Feat: Get block by id directly on promtool analyze & get latest block if ID not provided (#12031)
* feat: analyze latest block or block by ID in CLI (promtool)

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>

* address remarks

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

* address latest review comments

Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>

---------

Signed-off-by: nidhey27 <nidhey.indurkar@infracloud.io>
Signed-off-by: nidhey60@gmail.com <nidhey.indurkar@infracloud.io>
2023-06-01 17:13:09 +05:30
Alan Protasio 73078bf738
Opmizing Group Regex (#12375)
Signed-off-by: Alan Protasio <alanprot@gmail.com>
2023-05-30 13:49:22 +02:00
Julien Pivotto 6f97641a51
Merge pull request #12380 from mmorel-35/patch-2
ci(lint): enable predeclared linter
2023-05-28 14:43:29 +02:00
Justin Lei e73d8b2084 Also pass chunkOpts into appendPreprocessor
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-05-25 13:37:18 -07:00
Justin Lei 4c4454e4c9 Group args to append to memSeries in chunkOpts
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-05-25 13:12:46 -07:00
Justin Lei 89af351730
Remove samplesPerChunk from memSeries (#12390)
Signed-off-by: Justin Lei <justin.lei@grafana.com>
2023-05-25 11:18:41 +02:00
Jeanette Tan dd172440e5 Merge remote-tracking branch 'mine/fix-default-samples-per-chunk' into zenador/sync-upstream-22-may-2023 2023-05-24 19:32:04 +08:00
zenador 37e5249e33
Use DefaultSamplesPerChunk in tsdb (#12387)
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-05-24 13:00:21 +02:00
Jeanette Tan 02e113d03f Use DefaultSamplesPerChunk in tsdb
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-05-24 16:44:53 +08:00
Jeanette Tan 9f1baaaa38 Fix errors from merge 2023-05-23 01:10:09 +08:00
Jeanette Tan 1be0816b46 Merge remote-tracking branch 'upstream/main' 2023-05-23 00:20:36 +08:00
Baskar Shanmugam 905a0bd63a
Added 'limit' query parameter support to /api/v1/status/tsdb endpoint (#12336)
* Added 'topN' query parameter support to /api/v1/status/tsdb endpoint

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

* Updated query parameter for tsdb status to 'limit'

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

* Corrected Stats() parameter name from topN to limit

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

* Fixed p.Stats CI failure

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>

---------

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>
2023-05-22 14:37:07 +02:00
Alan Protasio 8c5d4b4add
Opmize MatchNotEqual (#12377)
Signed-off-by: Alan Protasio <alanprot@gmail.com>
2023-05-21 10:41:30 +02:00
Matthieu MOREL c8e7f95a3c ci(lint): enable predeclared linter
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-05-21 07:33:54 +00:00
George Krajcsovits 92d6980360
Fix populateWithDelChunkSeriesIterator and gauge histograms (#12330)
Use AppendableGauge to detect corrupt chunk with gauge histograms.
Detect if first sample is a gauge but the chunk is not set up to contain
gauge histograms.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2023-05-19 10:24:06 +02:00
Baskar Shanmugam f731a90a7f
Fix LabelValueStats in posting stats (#12342)
Problem:
LabelValueStats - This will provide a list of the label names and memory used in bytes.
It is calculated by adding the length of all values for a given label name.
But internally Prometheus stores the name and the value independently for each series.

Solution:
MemPostings struct maintains the values to seriesRef map which is used
to get the number of series which contains the label values.
Using that LabelValueStats is calculated as: seriesCnt * len(value
name)

Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>
2023-05-19 09:36:30 +02:00
Xiaochao Dong 80b7f73d26
Copy tombstone intervals to avoid race (#12245)
Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>
2023-05-17 15:15:12 +02:00
Björn Rabenstein 30e263cf96
Merge pull request #12357 from krajorama/fix-histogram-appendable-emptybucket
fix HistogramAppender.appendable segfault
2023-05-16 20:52:39 +02:00
Callum Styan 0d2108ad79
[tsdb] re-implement WAL watcher to read via a "notification" channel (#11949)
* WIP implement WAL watcher reading via notifications over a channel from
the TSDB code

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Notify via head appenders Commit (finished all WAL logging) rather than
on each WAL Log call

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix misspelled Notify plus add a metric for dropped Write notifications

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Update tests to handle new notification pattern

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* this test maybe needs more time on windows?

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* does this test need more time on windows as well?

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* read timeout is already a time.Duration

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* remove mistakenly commited benchmark data files

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* address some review feedback

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* fix missed changes from previous commit

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix issues from wrapper function

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* try fixing race condition in test by allowing tests to overwrite the
read ticker timeout instead of calling the Notify function

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* fix linting

Signed-off-by: Callum Styan <callumstyan@gmail.com>

---------

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2023-05-15 12:31:49 -07:00
György Krajcsovits c6618729c9 Fix HistogramAppender.Appendable array out of bound error
The code did not handle spans with 0 length properly.

Spans with length zero are now skipped in the comparison.
Span index check not done against length-1, since length is a unit32,
thus subtracting 1 leads to 2^32, not -1.

Fixes and unit tests for both integer and float histograms added.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2023-05-14 17:38:52 +02:00