prometheus/tsdb
Julien Pivotto ed623f69e2
tsdb: don't allow ingesting empty labelsets (#6891)
* tsdb: don't allow ingesting empty labelsets

When we ingest an empty labelset in the head, further blocks can not be
compacted, with the error:

```
level=error ts=2020-02-27T21:26:58.379Z caller=db.go:659 component=tsdb
msg="compaction failed" err="persist head block: write compaction:
add series: out-of-order series added with label set \"{}\" / prev:
\"{}\""
```

We should therefore reject those invalid empty labelsets upfront.

This can be reproduced with the following:

```
cat << END > prometheus.yml
scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 1s
    basic_auth:
      username: test
      password: test
    metric_relabel_configs:
    - regex: ".*"
      action: labeldrop

    static_configs:
    - targets:
      - 127.0.1.1:9090
END
./prometheus --storage.tsdb.min-block-duration=1m
```
And wait a few minutes.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-03-02 07:18:05 +00:00
..
chunkenc Specify that returned samples must be ordered by timestamp. (#6877) 2020-02-26 13:11:55 +00:00
chunks Refactor tsdb/chunks/chunks.go for future PRs (#6754) 2020-02-05 19:09:40 +05:30
cmd/tsdb Make head Postings only return series in time range 2020-02-20 22:41:46 +01:00
docs/format Stream symbols during compaction. (#6468) 2019-12-17 19:49:54 +00:00
encoding Avoid WriteAt for Postings. 2019-12-16 15:30:49 +00:00
errors Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
fileutil Add OpenMmapFileWithSize method (#6753) 2020-02-05 19:08:30 +05:30
goversion Fix some typo in comments (#6730) 2020-01-31 12:11:52 +05:30
index tsdb: writePostingsToTmpFiles returns nil instead of err (#6618) 2020-01-13 22:40:12 +00:00
record Spelling (#6517) 2020-01-02 15:54:09 +01:00
test Port tsdb to use pkg/labels. (#6326) 2019-11-18 11:53:33 -08:00
testdata Handle V1 indexes, some of which have unsorted posting offset tables. (#6564) 2020-01-06 14:06:11 +00:00
tombstones Bring back tombstones to Head block (#6542) 2020-01-20 21:08:00 +05:30
tsdbutil Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
wal Make head Postings only return series in time range 2020-02-20 22:41:46 +01:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block.go Make head Postings only return series in time range 2020-02-20 22:41:46 +01:00
block_test.go Unify Iterator interfaces. All point to storage now. 2020-02-17 18:03:54 +00:00
CHANGELOG.md Spelling (#6517) 2020-01-02 15:54:09 +01:00
compact.go Use a cryptographically random generator for ULID 2020-02-25 12:12:03 +01:00
compact_test.go Make head Postings only return series in time range 2020-02-20 22:41:46 +01:00
db.go Moved readyStorage to main. 2020-02-17 18:03:57 +00:00
db_test.go tsdb: don't allow ingesting empty labelsets (#6891) 2020-03-02 07:18:05 +00:00
head.go tsdb: don't allow ingesting empty labelsets (#6891) 2020-03-02 07:18:05 +00:00
head_bench_test.go Make head Postings only return series in time range 2020-02-20 22:41:46 +01:00
head_test.go Do not attempt isolation for appendID == 0 2020-03-01 02:48:35 +01:00
isolation.go Do not attempt isolation for appendID == 0 2020-03-01 02:48:35 +01:00
mocks_test.go Make head Postings only return series in time range 2020-02-20 22:41:46 +01:00
querier.go Make head Postings only return series in time range 2020-02-20 22:18:42 +01:00
querier_bench_test.go Make head Postings only return series in time range 2020-02-20 22:41:46 +01:00
querier_test.go Make head Postings only return series in time range 2020-02-20 22:41:46 +01:00
README.md Edit TSDB README badges 2019-10-24 15:35:47 +05:30
repair.go tsdb: add block meta version constant (#5994) 2019-09-09 12:28:01 +03:00
repair_test.go Port tsdb to use pkg/labels. (#6326) 2019-11-18 11:53:33 -08:00
tsdbblockutil.go Unify Iterator interfaces. All point to storage now. 2020-02-17 18:03:54 +00:00
wal.go Spelling (#6517) 2020-01-02 15:54:09 +01:00
wal_test.go Spelling (#6517) 2020-01-02 15:54:09 +01:00

TSDB

GoDoc

This repository contains the Prometheus storage layer that is used in its 2.x releases.

A writeup of its design can be found here.

Based on the Gorilla TSDB white papers.

Video: Storing 16 Bytes at Scale from PromCon 2017.

See also the format documentation.