On a 32 bit architecture the size of int is 32 bits. Thus converting from
int64, uint64 can overflow it and flip the sign.
Try for yourself in playground:
package main
import "fmt"
func main() {
x := int64(0x1F0000001)
y := int64(1)
z := int32(x - y) // numerically this is 0x1F0000000
fmt.Printf("%v\n", z)
}
Prints -268435456 as if x was smaller.
Followup to #12650
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
We need to make sure that `tsdb_errors.NewMulti` handles the errors.Is()
calls properly, like it's done in grafana/dskit.
Also we need to check that `errors.Is(err, context.Canceled)`, not that
`err == context.Canceled`.
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
We need to make sure that `tsdb_errors.NewMulti` handles the errors.Is()
calls properly, like it's done in grafana/dskit.
Also we need to check that `errors.Is(err, context.Canceled)`, not that
`err == context.Canceled`.
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Instead of relying on `labels.Hash()`, which may change, have the
caller pass in a shard function if required.
For most purposes `tsdb.Options.ShardFunc` is used, but the compactor
may be created independently so `NewLeveledCompactorWithChunkSize` also
takes a shard function parameter.
Regular Prometheus, which does not use block sharding, will have this
parameter as nil.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Patterned after `Chunk.Iterator()`: pass the old iterator in so it
can be re-used to avoid allocating a new object.
(This commit does not do any re-use; it is just changing all the method
signatures so re-use is possible in later commits.)
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Previously, the maxTime wasn't updated properly in case of a recoding
happening.
My apologies for reformatting many lines for line length. During the
bug hunt, I tried to make things more readable in a reasonably wide
editor window.
Signed-off-by: beorn7 <beorn@grafana.com>
* Avoid gaps in in-order data after restart with out-of-order enabled
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* Fix tests, do the temporary patch only if OOO is enabled
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* Avoid Peter's confusion
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* Use latest OutOfOrderTimeWindow
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
This implementation is based on this design doc:
https://docs.google.com/document/d/1Kppm7qL9C-BJB1j6yb6-9ObG3AbdZnFUBYPNNWwDBYM/edit?usp=sharing
This commit adds support to accept out-of-order ("OOO") sample into the TSDB
up to a configurable time allowance. If OOO is enabled, overlapping querying
are automatically enabled.
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Co-authored-by: Jesus Vazquez <jesus.vazquez@grafana.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Co-authored-by: Dieter Plaetinck <dieter@grafana.com>
* Tool for CLI compactions.
* Use concurrency when populating symbols for multiple blocks.
* Use concurrency when writing to multiple output blocks.
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
* Add log to debug out-of-order chunks when compacting
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Revert test changes
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Do not use golang 1.17 specific functions
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* TSDB: demistify seriesRefs and ChunkRefs
The TSDB package contains many types of series and chunk references,
all shrouded in uint types. Often the same uint value may
actually mean one of different types, in non-obvious ways.
This PR aims to clarify the code and help navigating to relevant docs,
usage, etc much quicker.
Concretely:
* Use appropriately named types and document their semantics and
relations.
* Make multiplexing and demuxing of types explicit
(on the boundaries between concrete implementations and generic
interfaces).
* Casting between different types should be free. None of the changes
should have any impact on how the code runs.
TODO: Implement BlockSeriesRef where appropriate (for a future PR)
Signed-off-by: Dieter Plaetinck <dieter@grafana.com>
* feedback
Signed-off-by: Dieter Plaetinck <dieter@grafana.com>
* agent: demistify seriesRefs and ChunkRefs
Signed-off-by: Dieter Plaetinck <dieter@grafana.com>
* Allow to disable trimming when querying TSDB
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Addressed review comments
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Added unit test
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Renamed TrimDisabled to DisableTrimming
Signed-off-by: Marco Pracucci <marco@pracucci.com>
