Commit graph

22 commits

Author SHA1 Message Date
Lukasz Mierzwa 565c6fa704 Reduce stringlabels memory usage for common labels
stringlabels stores all time series labels as a single string using this format:

<length><name><length><value>[<length><name><length><value> ...]

So a label set for my_metric{job=foo, instance="bar", env="prod", blank=""} would be encoded as:

[8]__name__[9]my_metric[3]job[3]foo[8]instance[3]bar[3]env[4]prod[5]blank[0]

This is a huge improvement over 'classic' labels implementation that stores all label names & values as seperate strings.
There is some room for improvement though since some string are present more often than others.
For example __name__ will be present for all label sets of every time series we store in HEAD, eating 1+8=9 bytes.
Since __name__ is well known string we can try to use a single byte to store it in our encoded string, rather than repeat it in full each time.
To be able to store strings that are short cut into a single byte we need to somehow signal that to the reader of the encoded string, for that
we use the fact that zero length strings are rare and generaly not stored on time series. If we have an encoded string with zero length
then this will now signal that it represents a mapped value - to learn the true value of this string we need to read the next byte
which gives us index in a static mapping. That mapping must include empty string, so that we can still encode empty strings using this scheme.

Example of our mapping (minimal version):

0: ""
1: "__name__"
2: "instance"
3: "job"

With that mapping our example label set would be encoded as:

[0]1[9]mymetric[0]3[3]foo[0]2[3]bar[3]env[4]prod[5]blank[0]0

The tricky bit is how to populate this mapping with useful strings that will result in measurable memory savings.
This is further complicated by the fact that the mapping must remain static and cannot be modified during Prometheus lifetime.
We can use all the 255 slots we have inside our mapping byte with well known generic strings and that will
provide some measurable savings for all Prometheus users, and is essentially a slightly more compact stringlabels variant.
We could also allow users to pass in a list of well know strings via flags, which will allow Prometheus operators
to reduce memory usage for any labels if they know those are popular.
Third option is to discover most popular strings from TSDB or WAL on startup, but that's more complicated and
we might pick a list that would be the best set of mapped strings on startup, but after some time is no longer
the best set.

Benchmark results:

goos: linux
goarch: amd64
pkg: github.com/prometheus/prometheus/model/labels
cpu: 13th Gen Intel(R) Core(TM) i7-13800H
                                                 │   main.txt   │                new1.txt                │
                                                 │    sec/op    │    sec/op      vs base                 │
String-20                                           863.8n ± 4%    873.0n ±  4%         ~ (p=0.353 n=10)
Labels_Get/with_5_labels/first_label/get-20         4.763n ± 1%    5.035n ±  0%    +5.72% (p=0.000 n=10)
Labels_Get/with_5_labels/first_label/has-20         3.439n ± 0%    3.967n ±  0%   +15.37% (p=0.000 n=10)
Labels_Get/with_5_labels/middle_label/get-20        7.077n ± 1%    9.588n ±  1%   +35.47% (p=0.000 n=10)
Labels_Get/with_5_labels/middle_label/has-20        5.166n ± 0%    6.990n ±  1%   +35.30% (p=0.000 n=10)
Labels_Get/with_5_labels/last_label/get-20          9.181n ± 1%   12.970n ±  1%   +41.26% (p=0.000 n=10)
Labels_Get/with_5_labels/last_label/has-20          8.101n ± 1%   11.640n ±  1%   +43.69% (p=0.000 n=10)
Labels_Get/with_5_labels/not-found_label/get-20     3.974n ± 0%    4.768n ±  0%   +19.98% (p=0.000 n=10)
Labels_Get/with_5_labels/not-found_label/has-20     3.974n ± 0%    5.033n ±  0%   +26.65% (p=0.000 n=10)
Labels_Get/with_10_labels/first_label/get-20        4.761n ± 0%    5.042n ±  0%    +5.90% (p=0.000 n=10)
Labels_Get/with_10_labels/first_label/has-20        3.442n ± 0%    3.972n ±  0%   +15.40% (p=0.000 n=10)
Labels_Get/with_10_labels/middle_label/get-20       10.62n ± 1%    14.85n ±  1%   +39.83% (p=0.000 n=10)
Labels_Get/with_10_labels/middle_label/has-20       9.360n ± 1%   13.375n ±  0%   +42.90% (p=0.000 n=10)
Labels_Get/with_10_labels/last_label/get-20         18.19n ± 1%    22.00n ±  0%   +20.97% (p=0.000 n=10)
Labels_Get/with_10_labels/last_label/has-20         16.51n ± 0%    20.50n ±  1%   +24.14% (p=0.000 n=10)
Labels_Get/with_10_labels/not-found_label/get-20    3.985n ± 0%    4.768n ±  0%   +19.62% (p=0.000 n=10)
Labels_Get/with_10_labels/not-found_label/has-20    3.973n ± 0%    5.045n ±  0%   +26.97% (p=0.000 n=10)
Labels_Get/with_30_labels/first_label/get-20        4.773n ± 0%    5.050n ±  1%    +5.80% (p=0.000 n=10)
Labels_Get/with_30_labels/first_label/has-20        3.443n ± 1%    3.976n ±  2%   +15.50% (p=0.000 n=10)
Labels_Get/with_30_labels/middle_label/get-20       31.93n ± 0%    43.50n ±  1%   +36.21% (p=0.000 n=10)
Labels_Get/with_30_labels/middle_label/has-20       30.53n ± 0%    41.75n ±  1%   +36.75% (p=0.000 n=10)
Labels_Get/with_30_labels/last_label/get-20        106.55n ± 0%    71.17n ±  0%   -33.21% (p=0.000 n=10)
Labels_Get/with_30_labels/last_label/has-20        104.70n ± 0%    69.21n ±  1%   -33.90% (p=0.000 n=10)
Labels_Get/with_30_labels/not-found_label/get-20    3.976n ± 1%    4.772n ±  0%   +20.03% (p=0.000 n=10)
Labels_Get/with_30_labels/not-found_label/has-20    3.974n ± 0%    5.032n ±  0%   +26.64% (p=0.000 n=10)
Labels_Equals/equal-20                              2.382n ± 0%    2.446n ±  0%    +2.67% (p=0.000 n=10)
Labels_Equals/not_equal-20                         0.2741n ± 2%   0.2662n ±  2%    -2.88% (p=0.001 n=10)
Labels_Equals/different_sizes-20                   0.2762n ± 3%   0.2652n ±  0%    -3.95% (p=0.000 n=10)
Labels_Equals/lots-20                               2.381n ± 0%    2.386n ±  1%    +0.23% (p=0.011 n=10)
Labels_Equals/real_long_equal-20                    6.087n ± 1%    5.558n ±  1%    -8.70% (p=0.000 n=10)
Labels_Equals/real_long_different_end-20            5.030n ± 0%    4.699n ±  0%    -6.57% (p=0.000 n=10)
Labels_Compare/equal-20                             4.814n ± 1%    4.777n ±  0%    -0.77% (p=0.000 n=10)
Labels_Compare/not_equal-20                         17.55n ± 8%    20.92n ±  1%   +19.24% (p=0.000 n=10)
Labels_Compare/different_sizes-20                   3.711n ± 1%    3.707n ±  0%         ~ (p=0.224 n=10)
Labels_Compare/lots-20                              27.09n ± 3%    28.73n ±  2%    +6.05% (p=0.000 n=10)
Labels_Compare/real_long_equal-20                   27.91n ± 3%    15.67n ±  1%   -43.86% (p=0.000 n=10)
Labels_Compare/real_long_different_end-20           33.92n ± 1%    35.35n ±  1%    +4.22% (p=0.000 n=10)
Labels_Hash/typical_labels_under_1KB-20             59.63n ± 0%    59.67n ±  0%         ~ (p=0.897 n=10)
Labels_Hash/bigger_labels_over_1KB-20               73.42n ± 1%    73.81n ±  1%         ~ (p=0.342 n=10)
Labels_Hash/extremely_large_label_value_10MB-20     720.3µ ± 2%    715.2µ ±  3%         ~ (p=0.971 n=10)
Builder-20                                          371.6n ± 4%   1191.0n ±  3%  +220.46% (p=0.000 n=10)
Labels_Copy-20                                      85.52n ± 4%    53.90n ± 48%   -36.97% (p=0.000 n=10)
geomean                                             13.26n         14.68n         +10.71%

                                                 │   main.txt   │               new1.txt               │
                                                 │     B/op     │    B/op     vs base                  │
String-20                                          240.0 ± 0%     240.0 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/first_label/get-20        0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/first_label/has-20        0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/middle_label/get-20       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/middle_label/has-20       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/last_label/get-20         0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/last_label/has-20         0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/not-found_label/get-20    0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/not-found_label/has-20    0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/first_label/get-20       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/first_label/has-20       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/middle_label/get-20      0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/middle_label/has-20      0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/last_label/get-20        0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/last_label/has-20        0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/not-found_label/get-20   0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/not-found_label/has-20   0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/first_label/get-20       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/first_label/has-20       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/middle_label/get-20      0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/middle_label/has-20      0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/last_label/get-20        0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/last_label/has-20        0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/not-found_label/get-20   0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/not-found_label/has-20   0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Equals/equal-20                             0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Equals/not_equal-20                         0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Equals/different_sizes-20                   0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Equals/lots-20                              0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Equals/real_long_equal-20                   0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Equals/real_long_different_end-20           0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Compare/equal-20                            0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Compare/not_equal-20                        0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Compare/different_sizes-20                  0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Compare/lots-20                             0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Compare/real_long_equal-20                  0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Compare/real_long_different_end-20          0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Hash/typical_labels_under_1KB-20            0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Hash/bigger_labels_over_1KB-20              0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Labels_Hash/extremely_large_label_value_10MB-20    0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
Builder-20                                         224.0 ± 0%     192.0 ± 0%  -14.29% (p=0.000 n=10)
Labels_Copy-20                                     224.0 ± 0%     192.0 ± 0%  -14.29% (p=0.000 n=10)
geomean                                                       ²                -0.73%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                                 │   main.txt   │              new1.txt               │
                                                 │  allocs/op   │ allocs/op   vs base                 │
String-20                                          1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/first_label/get-20        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/first_label/has-20        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/middle_label/get-20       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/middle_label/has-20       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/last_label/get-20         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/last_label/has-20         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/not-found_label/get-20    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_5_labels/not-found_label/has-20    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/first_label/get-20       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/first_label/has-20       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/middle_label/get-20      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/middle_label/has-20      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/last_label/get-20        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/last_label/has-20        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/not-found_label/get-20   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_10_labels/not-found_label/has-20   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/first_label/get-20       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/first_label/has-20       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/middle_label/get-20      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/middle_label/has-20      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/last_label/get-20        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/last_label/has-20        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/not-found_label/get-20   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Get/with_30_labels/not-found_label/has-20   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Equals/equal-20                             0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Equals/not_equal-20                         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Equals/different_sizes-20                   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Equals/lots-20                              0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Equals/real_long_equal-20                   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Equals/real_long_different_end-20           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Compare/equal-20                            0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Compare/not_equal-20                        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Compare/different_sizes-20                  0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Compare/lots-20                             0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Compare/real_long_equal-20                  0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Compare/real_long_different_end-20          0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Hash/typical_labels_under_1KB-20            0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Hash/bigger_labels_over_1KB-20              0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Hash/extremely_large_label_value_10MB-20    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Builder-20                                         1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Labels_Copy-20                                     1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                                       ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>
2025-02-20 12:06:33 +00:00
Bryan Boreham ac4f8a5e23
[ENHANCEMENT] TSDB: Improve calculation of space used by labels (#13880)
Some checks are pending
CI / Go tests (push) Waiting to run
CI / More Go tests (push) Waiting to run
CI / Go tests with previous Go version (push) Waiting to run
CI / UI tests (push) Waiting to run
CI / Go tests on Windows (push) Waiting to run
CI / Mixins tests (push) Waiting to run
CI / Build Prometheus for common architectures (0) (push) Waiting to run
CI / Build Prometheus for common architectures (1) (push) Waiting to run
CI / Build Prometheus for common architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (0) (push) Waiting to run
CI / Build Prometheus for all architectures (1) (push) Waiting to run
CI / Build Prometheus for all architectures (10) (push) Waiting to run
CI / Build Prometheus for all architectures (11) (push) Waiting to run
CI / Build Prometheus for all architectures (2) (push) Waiting to run
CI / Build Prometheus for all architectures (3) (push) Waiting to run
CI / Build Prometheus for all architectures (4) (push) Waiting to run
CI / Build Prometheus for all architectures (5) (push) Waiting to run
CI / Build Prometheus for all architectures (6) (push) Waiting to run
CI / Build Prometheus for all architectures (7) (push) Waiting to run
CI / Build Prometheus for all architectures (8) (push) Waiting to run
CI / Build Prometheus for all architectures (9) (push) Waiting to run
CI / Report status of build Prometheus for all architectures (push) Blocked by required conditions
CI / Check generated parser (push) Waiting to run
CI / golangci-lint (push) Waiting to run
CI / fuzzing (push) Waiting to run
CI / codeql (push) Waiting to run
CI / Publish main branch artifacts (push) Blocked by required conditions
CI / Publish release artefacts (push) Blocked by required conditions
CI / Publish UI on npm Registry (push) Blocked by required conditions
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
* [ENHANCEMENT] TSDB: Improve calculation of space used by labels

The labels for each series in the Head take up some some space in the
Postings index, but far more space in the `memSeries` structure.

Instead of having the Postings index calculate this overhead, which is
a layering violation, have the caller pass in a function to do it.

Provide three implementations of this function for the three Labels
versions.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-12-16 09:42:52 +00:00
Bryan Boreham 31c5760551
Neater string vs byte-slice conversions (#14425)
Some checks failed
CI / Go tests (push) Has been cancelled
CI / More Go tests (push) Has been cancelled
CI / Go tests with previous Go version (push) Has been cancelled
CI / UI tests (push) Has been cancelled
CI / Go tests on Windows (push) Has been cancelled
CI / Mixins tests (push) Has been cancelled
CI / Build Prometheus for common architectures (0) (push) Has been cancelled
CI / Build Prometheus for common architectures (1) (push) Has been cancelled
CI / Build Prometheus for common architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (0) (push) Has been cancelled
CI / Build Prometheus for all architectures (1) (push) Has been cancelled
CI / Build Prometheus for all architectures (10) (push) Has been cancelled
CI / Build Prometheus for all architectures (11) (push) Has been cancelled
CI / Build Prometheus for all architectures (2) (push) Has been cancelled
CI / Build Prometheus for all architectures (3) (push) Has been cancelled
CI / Build Prometheus for all architectures (4) (push) Has been cancelled
CI / Build Prometheus for all architectures (5) (push) Has been cancelled
CI / Build Prometheus for all architectures (6) (push) Has been cancelled
CI / Build Prometheus for all architectures (7) (push) Has been cancelled
CI / Build Prometheus for all architectures (8) (push) Has been cancelled
CI / Build Prometheus for all architectures (9) (push) Has been cancelled
CI / Check generated parser (push) Has been cancelled
CI / golangci-lint (push) Has been cancelled
CI / fuzzing (push) Has been cancelled
CI / codeql (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
CI / Report status of build Prometheus for all architectures (push) Has been cancelled
CI / Publish main branch artifacts (push) Has been cancelled
CI / Publish release artefacts (push) Has been cancelled
CI / Publish UI on npm Registry (push) Has been cancelled
unsafe.Slice and unsafe.StringData were added in Go 1.20

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-09-21 12:19:21 +02:00
Bryan Boreham d84282b105 Labels: use single byte as separator - small speedup
Since `seps` is a variable, `seps[0]` has to be bounds-checked every
time. Replacing with a constant everywhere it is used skips this
overhead.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-07-15 09:47:16 +01:00
Oleg Zaytsev 4f78cc809c
Refactor toNormalisedLower: shorter and slightly faster. (#14299)
Refactor toNormalisedLower: shorter and slightly faster

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-06-18 09:57:37 +00:00
Oleg Zaytsev d12e785075
Improve readability
As suggested by @bboreham

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-03-18 11:16:09 +01:00
Oleg Zaytsev 9699598952
Improve Labels.Compare performance w/stringlabels
I was bored on a train and I spent some amount of time trying to scratch
some nanoseconds off the Labels.Compare when running with stringlabels.

I would be ashamed to admit the real amount of time I spent on it.

The worst thing is, I can't really explain why this is performing so
much better, and someone should re-run the benchmarks on their machine
to confirm that it's not something related to general relativity because
the train is moving. I also added some extra real-life benchmark cases
with longer labelsets (these aren't the longest we have in production,
but kubernetes labelsets are fairly common in Prometheus so I thought it
would be nice to have them).

My benchmarks show this diff:

goos: darwin
goarch: arm64
pkg: github.com/prometheus/prometheus/model/labels
                                       │     old     │                 new                 │
                                       │   sec/op    │   sec/op     vs base                │
Labels_Compare/equal                     5.898n ± 0%   5.875n ± 1%   -0.40% (p=0.037 n=10)
Labels_Compare/not_equal                 11.78n ± 2%   11.01n ± 1%   -6.54% (p=0.000 n=10)
Labels_Compare/different_sizes           4.959n ± 1%   4.906n ± 2%   -1.05% (p=0.050 n=10)
Labels_Compare/lots                      21.32n ± 0%   17.54n ± 5%  -17.75% (p=0.000 n=10)
Labels_Compare/real_long_equal           15.06n ± 1%   14.92n ± 0%   -0.93% (p=0.000 n=10)
Labels_Compare/real_long_different_end   25.20n ± 0%   24.43n ± 0%   -3.04% (p=0.000 n=10)
geomean                                  11.86n        11.25n        -5.16%

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-03-17 17:08:06 +01:00
machine424 f477e0539a
Move from golang.org/x/exp/slices into slices now that we only support Go >= 1.21
Prevent adding back golang.org/x/exp/slices.

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-02-28 14:54:53 +01:00
Bryan Boreham 55e7de04f8 model/labels (stringlabels): use strings.Clone
Suggestion from @colega.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
Bryan Boreham 28191109a8 Labels: add fake versions of SymbolTable apis
So we can use them where necessary for internlabels implementation.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham d1af84f6ee Labels: move Builder and Reset out of common
New internstrings implementation is different.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
Bryan Boreham 14b4fbc2ff labels: make InternStrings a no-op for stringlabels version
The current implementation of `InternStrings` will only save memory
when the whole set of labels is identical to one already seen, and this
cannot happen in the one place it is called from in Prometheus,
remote-write, which already detects identical series.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-01-29 18:54:54 +00:00
Bryan Boreham 74b73d1e2c
Labels: Add DropMetricName function, used in PromQL (#13446)
This function is called very frequently when executing PromQL functions,
and we can do it much more efficiently inside Labels.

In the common case that `__name__` comes first in the labels, we simply
re-point to start at the next label, which is nearly free.

`DropMetricName` is now so cheap I removed the cache - benchmarks show
everything still goes faster.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-01-25 11:48:49 +01:00
Bryan Boreham a3e02f35d6 labels: extract common code between slice and stringlabels
This reduces bulk and should avoid issues if a fix is made in one file
and not the other.

A few methods now call `Range()` instead of `range`, but nothing
performance-sensitive.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-17 18:20:03 +00:00
Bryan Boreham 1bfb3ed062
Labels: reduce allocations when creating from TSDB WAL (#13044)
* Labels: reduce allocations when creating from TSDB

When reading the WAL, by passing references into the buffer we can avoid
copying strings under `-tags stringlabels`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-11-14 11:36:35 +00:00
Goutham Veeramachaneni 86729d4d7b
Update exp package (#12650) 2023-09-21 22:53:51 +02:00
Bryan Boreham d6e1b1acdb
Merge pull request #12681 from prometheus/labels-unused-code
labels: remove some unused code
2023-08-14 15:48:17 +01:00
Bryan Boreham ce260b1fe1 labels: remove some unused code
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-08-13 15:14:53 +01:00
Bryan Boreham b5c6807fea Labels.Has quick check on first character
Exit early if we've gone past - labels are sorted in order.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-08-13 15:11:22 +01:00
Bryan Boreham e1115ae58d
labels: improve Get method for stringlabels build (#12485)
Inline one call to `decodeString`, and skip decoding the value string
until we find a match for the name.
Do a quick check on the first character in each string,
and exit early if we've gone past - labels are sorted in order.

Also improve tests and benchmark:
* labels: test Get with varying lengths - it's not typical for Prometheus labels to all be the same length.
* extend benchmark with label not found

---------

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-06-26 18:35:22 +01:00
Bryan Boreham 87d08abe11
labels: faster Compare function when using -tags stringlabels (#12451)
Instead of unpacking every individual string, we skip to the point
where there is a difference, going 8 bytes at a time where possible.

Add benchmark for Compare; extend tests too.

---------

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com>
2023-06-20 20:58:47 +01:00
Oleg Zaytsev 6a18962cfa
mv labels_string.go labels_stringlabels.go (#12328)
This is a minor cosmetical change, but my IDE (and I guess many of them)
nests `labels_string.go` under `labels.go` because it assumes it's the
file generated by the `stringer` tool, which follows that naming
pattern.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2023-06-13 09:38:00 +01:00
Renamed from model/labels/labels_string.go (Browse further)