mirror of
https://github.com/prometheus/prometheus.git
synced 2024-11-15 18:14:06 -08:00
7a8bb8222c
A lot of this code was hacked together, literally during a hackathon. This commit intends not to change the code substantially, but just make the code obey the usual style practices. A (possibly incomplete) list of areas: * Generally address linter warnings. * The `pgk` directory is deprecated as per dev-summit. No new packages should be added to it. I moved the new `pkg/histogram` package to `model` anticipating what's proposed in #9478. * Make the naming of the Sparse Histogram more consistent. Including abbreviations, there were just too many names for it: SparseHistogram, Histogram, Histo, hist, his, shs, h. The idea is to call it "Histogram" in general. Only add "Sparse" if it is needed to avoid confusion with conventional Histograms (which is rare because the TSDB really has no notion of conventional Histograms). Use abbreviations only in local scope, and then really abbreviate (not just removing three out of seven letters like in "Histo"). This is in the spirit of https://github.com/golang/go/wiki/CodeReviewComments#variable-names * Several other minor name changes. * A lot of formatting of doc comments. For one, following https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences , but also layout question, anticipating how things will look like when rendered by `godoc` (even where `godoc` doesn't render them right now because they are for unexported types or not a doc comment at all but just a normal code comment - consistency is queen!). * Re-enabled `TestQueryLog` and `TestEndopints` (they pass now, leaving them disabled was presumably an oversight). * Bucket iterator for histogram.Histogram is now created with a method. * HistogramChunk.iterator now allows iterator recycling. (I think @dieterbe only commented it out because he was confused by the question in the comment.) * HistogramAppender.Append panics now because we decided to treat staleness marker differently. Signed-off-by: beorn7 <beorn@grafana.com>
56 lines
3.3 KiB
Markdown
56 lines
3.3 KiB
Markdown
# Chunks Disk Format
|
|
|
|
The following describes the format of a chunks file,
|
|
which is created in the `chunks/` directory of a block.
|
|
The maximum size per segment file is 512MiB.
|
|
|
|
Chunks in the files are referenced from the index by uint64 composed of
|
|
in-file offset (lower 4 bytes) and segment sequence number (upper 4 bytes).
|
|
|
|
```
|
|
┌──────────────────────────────┐
|
|
│ magic(0x85BD40DD) <4 byte> │
|
|
├──────────────────────────────┤
|
|
│ version(1) <1 byte> │
|
|
├──────────────────────────────┤
|
|
│ padding(0) <3 byte> │
|
|
├──────────────────────────────┤
|
|
│ ┌──────────────────────────┐ │
|
|
│ │ Chunk 1 │ │
|
|
│ ├──────────────────────────┤ │
|
|
│ │ ... │ │
|
|
│ ├──────────────────────────┤ │
|
|
│ │ Chunk N │ │
|
|
│ └──────────────────────────┘ │
|
|
└──────────────────────────────┘
|
|
```
|
|
|
|
|
|
# Chunk
|
|
|
|
```
|
|
┌───────────────┬───────────────────┬──────────────┬────────────────┐
|
|
│ len <uvarint> │ encoding <1 byte> │ data <bytes> │ CRC32 <4 byte> │
|
|
└───────────────┴───────────────────┴──────────────┴────────────────┘
|
|
```
|
|
|
|
## XOR chunk
|
|
|
|
TODO(beorn7): Add.
|
|
|
|
## Histogram chunk
|
|
|
|
TODO(beorn7): This is out of date. Update once settled on the (more or less) final format.
|
|
|
|
```
|
|
┌──────────────┬─────────────────┬──────────────────────────┬──────────────────────────┬──────────────┐
|
|
│ len <uint16> │ schema <varint> │ pos-spans <span-section> │ neg-spans <span-section> │ data <bytes> │
|
|
└──────────────┴─────────────────┴──────────────────────────┴──────────────────────────┴──────────────┘
|
|
|
|
span-section:
|
|
|
|
┌──────────────┬──────────────────┬──────────────────┬────────────┐
|
|
│ len <varint> │ length1 <varint> │ offset1 <varint> │ length2... │
|
|
└──────────────┴──────────────────┴──────────────────┴────────────┘
|
|
```
|