Commit graph

7569 commits

Author SHA1 Message Date
Fabian Reinartz 65581a3d46 promql: export SmapleStream 2016-12-24 11:29:39 +01:00
Fabian Reinartz 6315d00942 promql: export String value 2016-12-24 11:25:26 +01:00
Fabian Reinartz ac5d3bc05e promql: scalar T/V and Point 2016-12-24 11:23:06 +01:00
Fabian Reinartz 09666e2e2a promql: make scalar public 2016-12-24 10:44:04 +01:00
Fabian Reinartz b3f71df350 promql: make matrix exported 2016-12-24 10:42:54 +01:00
Fabian Reinartz a62df87022 promql: rename vector 2016-12-24 10:40:09 +01:00
Fabian Reinartz 0d0c5cfaf1 labels: add string constructor, expose matcher 2016-12-24 10:19:46 +01:00
Fabian Reinartz 15a931dbdb promql: migrate model types, use tsdb interfaces 2016-12-24 00:39:52 +01:00
Fabian Reinartz 8b84ee5ee6 storage: remove old storage
This removes all old storage files and only keeps interfaces
to still allow the code to compile.
2016-12-22 23:33:32 +01:00
Fabian Reinartz 313ab48b45 vendor: drop unused dependencies 2016-12-22 23:20:34 +01:00
Fabian Reinartz 11a731ba82 remote: remove hard-coded remote storages
This commit removes the flag-configured remote storage integrations
in favor of the generic remote write path.
2016-12-22 23:17:35 +01:00
Fabian Reinartz 787199a88e Fix erroneous value assignments 2016-12-22 20:57:00 +01:00
Fabian Reinartz 8aba95048a Rename to OpenHeadBlock 2016-12-22 20:00:24 +01:00
Fabian Reinartz cabd7e4ebc Add small testdata file 2016-12-22 19:15:37 +01:00
Fabian Reinartz e7edae39b2 Use page buffered writer for WAL 2016-12-22 16:14:34 +01:00
Fabian Reinartz 9c6a72aadd Load head with WALs correctly 2016-12-22 15:54:39 +01:00
Fabian Reinartz 1dde3b6d31 Add WAL decoder+loading and benchmarks 2016-12-22 15:18:33 +01:00
Brian Brazil 93b70ee4ea Evict chunk descs of all unloaded chunks during maintenance. (#2297)
Keeping these around has two problems:
1) Each desc takes 64 bytes, 10 of them is 640B. This is a lot of
overhead on a 1024 byte chunk.
2) It can take well over a week to reach a point where this and thus
Prometheus memory usage as a whole enters steady state. This makes RAM
estimation very hard for users, and makes it difficult to investigate
things like memory fragmentation.

Instead we'll wipe them during each memory series maintenance cycle, and
if a query pulls them in they'll hang around as cache until the next
cycle.
2016-12-22 13:49:03 +00:00
Fabian Reinartz 0b8c77361e Add initial WAL writing 2016-12-22 12:05:24 +01:00
Fabian Reinartz 2a825f6c28 Consolidate mem index into HeadBlock 2016-12-22 01:12:28 +01:00
Fabian Reinartz 869cccf080 Test and fixes for buffered iterator 2016-12-21 16:06:33 +01:00
Fabian Reinartz 0a94f58f1a Fix test import of labels, simplify constructor names 2016-12-21 15:12:26 +01:00
Brian Brazil bed4635802 Use irate consistently in console template examples. (#2296)
I must have forgotten my 'g' when switching these.
2016-12-21 13:19:23 +00:00
Fabian Reinartz da2beb3e6d Fix zero division, add buffer series iterator 2016-12-21 13:04:51 +01:00
Fabian Reinartz d6d03a966f Merge pull request #2295 from prometheus/fast-path-remote
Don't clone the metric if there's no remote writes.
2016-12-21 12:36:41 +01:00
Brian Brazil 1b8a474612 Don't clone the metric if there's no remote writes.
The metric clone can't be further optimised, and is a
non-trivial memory allocation cost so fast path it
if there's no remote writes configured.
2016-12-21 11:34:48 +00:00
Brian Brazil 6c07453ec1 Only clone the metric in the one place relabelling needs it. (#2292)
This cuts ~17% off memory allocations related to ingesting data
in a basic setup.
2016-12-21 10:00:33 +00:00
Fabian Reinartz dbca3453fb Add label clone benchmark 2016-12-21 10:37:38 +01:00
Fabian Reinartz ede733ab6c Extract labels package 2016-12-21 09:39:01 +01:00
Fabian Reinartz ee217adc7e Redfine append interface, remove old Prometheus storage from bench 2016-12-21 00:02:37 +01:00
Fabian Reinartz cddc29fa17 Fix labels comparison, fetch correct labels 2016-12-20 14:54:52 +01:00
Fabian Reinartz ce7f4106c2 Reda correct label number, fix buffered iterator panic 2016-12-20 14:21:50 +01:00
Fabian Reinartz d9ca4b47f5 Fix offset errors, fix persisted postings order 2016-12-20 13:14:55 +01:00
Fabian Reinartz 1b23d62e3f Properly close files before reopening 2016-12-19 22:37:03 +01:00
Fabian Reinartz 00a503129b Use contextualized and traced errors in reader 2016-12-19 22:29:49 +01:00
Fabian Reinartz 282d9ae6e2 Implement label value queries in all layers. 2016-12-19 12:26:25 +01:00
Fabian Reinartz aabb21f4b9 Add shard series set test 2016-12-19 11:44:11 +01:00
Brian Brazil 2e3b42ad6c Correctly handle the end time being 0 in the URL. (#2290) 2016-12-18 19:30:52 +00:00
Fabian Reinartz bad93d8d57 Extract head serialization into Head method 2016-12-18 14:43:27 +01:00
Brian Brazil f421ce0636 Remove label from prometheus_target_skipped_scrapes_total (#2289)
This avoids it not being intialised, and breaking out by
interval wasn't partiuclarly useful.

Fixes #2269
2016-12-16 18:00:52 +00:00
Brian Brazil 30448286c7 Add sample_limit to scrape config.
This imposes a hard limit on the number of samples ingested from the
target. This is counted after metric relabelling, to allow dropping of
problemtic metrics.

This is intended as a very blunt tool to prevent overload due to
misbehaving targets that suddenly jump in sample count (e.g. adding
a label containing email addresses).

Add metric to track how often this happens.

Fixes #2137
2016-12-16 15:10:09 +00:00
Fabian Reinartz b08f82fa4e Pre-select relevant chunks on series access.
This adds interval metadata to indexed chunks. The queried interval
is used to filter chunks when queried from the index to save
unnecessary accesses of the chunks file.

This is especially relevant for series that come and go often and larger
files.
2016-12-16 12:13:17 +01:00
Fabian Reinartz bd77103a49 Add stats serialization, load querier of all blocks 2016-12-15 16:14:33 +01:00
Fabian Reinartz 1a35e54450 Add chained iterator, skipping seek added 2016-12-15 15:23:15 +01:00
Björn Rabenstein f3f798fbcf Merge pull request #2283 from tcolgate/ignoredots
ignore dotfiles in data directory
2016-12-15 13:32:03 +01:00
Tristan Colgate 30be8e0b8a ignore dotfiles in data directory 2016-12-15 11:48:23 +00:00
Fabian Reinartz 205edd2da9 Add byte/string equality check benchmark 2016-12-15 12:15:54 +01:00
Fabian Reinartz b2f1db5666 Add unsafe string and slice conversions 2016-12-15 11:56:41 +01:00
Fabian Reinartz 9ceed5378e Fix missing bound checks, off-by-ones, typos 2016-12-15 11:24:20 +01:00
Fabian Reinartz 5424a0cf75 Rename SeriesShard to Shard 2016-12-15 08:36:09 +01:00