Commit graph

565 commits

Author SHA1 Message Date
Krasi Georgiev 9c61f0e8a0 scrape pool doesn't rely on context as Stop() needs to be blocking to prevent Scrape loops trying to write to a closed TSDB storage. 2017-12-18 17:22:49 +00:00
Krasi Georgiev e405e2f1ea refactored discovery 2017-12-18 17:22:49 +00:00
conorbroderick 8b6f3a1bd3 Added service discovery view showing labels before and after relabelling 2017-12-13 15:46:11 +00:00
Krasi Georgiev 08a012ccd7 Fix for #3312 sigpanic in TargetManager (#3420) 2017-11-07 15:56:00 +00:00
Julius Volz 099df0c5f0 Migrate "golang.org/x/net/context" -> "context" (#3333)
In some places, where ctxhttp or gRPC are concerned, we still need to use the
old contexts.
2017-10-24 21:21:42 -07:00
Krasi Georgiev 55a1311f6a fix golint nits (#3291) 2017-10-20 14:59:53 +01:00
Brian Brazil 1545842704 TrimPrefix removes a prefix, TrimLeft removes chars 2017-10-09 19:44:03 +01:00
Marc Sluiter 6a633eece1 Added go-conntrack for monitoring http connections (#3241)
Added metrics for in- and outgoing traffic with go-conntrack.
2017-10-06 11:22:19 +01:00
Tobias Schmidt 40c278ee2d Send a HTTP Accept header when scraping 2017-09-25 14:51:29 +02:00
Fabian Reinartz 249d69b513 Merge pull request #3186 from prometheus/startweb
web: start web handler while TSDB is starting up
2017-09-21 09:53:03 +02:00
Fabian Reinartz 7b02bfee0a web: start web handler while TSDB is starting up 2017-09-20 15:03:19 +02:00
Fabian Reinartz 437f51a85f Fix cache maintenance on changing metric representations
We were not properly maintaining the scrape cache when the same metric
was exposed with a different string representation.
This overall reduces the scraping cache's complexity, which fixes the
issue and saves about 10% of memory in a scraping-only Prometheus
instance.
2017-09-19 15:03:27 +02:00
Goutham Veeramachaneni 3f0267c548 Merge branch 'dev-2.0' into go-kit/log
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-15 23:15:27 +05:30
Fabian Reinartz 1121b9f7d4 retrieval: cache dropped series, mutate labels in place 2017-09-14 08:36:19 +02:00
Fabian Reinartz d21f149745 *: migrate to go-kit/log 2017-09-08 22:01:51 +05:30
Fabian Reinartz 5bed8af4cb retrieval: pool scrape buffers
This adds a bucketed buffer pool to the scrapers so we don't have to
allocate a new buffer on each scrape or hold it fixed to the scrape
loop.

The latter can consume significant amounts of unused memory, e.g. 4GB
when scraping 2MB /metrics from 2000 targets.
2017-09-07 14:43:21 +02:00
Fabian Reinartz a8887f46dc Merge branch 'dev-2.0' of github.com:prometheus/prometheus into dev-2.0 2017-09-07 14:15:12 +02:00
Fabian Reinartz 0efecea6d4 Adapt storage APIs to uint64 references 2017-09-07 14:14:41 +02:00
Krasi Georgiev 153cb0cbe3 scraping errors will show in the log when debug mode is enabled (#3135)
Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2017-09-05 11:55:14 +01:00
Fabian Reinartz 9516d04472 util: Add idle timeout for scrape connections 2017-08-10 14:47:51 +02:00
Fabian Reinartz 4d3d8ee229 Merge pull request #2850 from tomwilkie/dev-2.0-remote
Remote APIs for v2
2017-08-03 13:39:09 +02:00
Edward Marshall c490725ac9 Additional targetScrapeSample metrics (#3018) 2017-08-02 13:10:18 +01:00
Tom Wilkie 1f3b59ccf5 s/met/lset/ 2017-07-18 11:42:29 +01:00
Tom Wilkie 014bd31a86 Remove unnecessary whitespace changes, add comment. 2017-07-13 11:26:46 +01:00
Tom Wilkie 2ac1809a5b Get label set from cache in addReportSample. 2017-07-12 22:09:16 +01:00
Tom Wilkie 240feb313b Don't regenerate label set for cached values. 2017-07-12 15:54:38 +01:00
Tom Wilkie db8128ceeb Add label set as first parameter to AddFast, ingored by TSDB adapter. 2017-07-12 15:20:12 +01:00
Fabian Reinartz dba7586671 Merge branch 'master' into dev-2.0 2017-07-11 17:22:14 +02:00
Goutham Veeramachaneni 243419c007 Return tsdb.ErrOutOfBounds as storage.ErrOutOfBounds
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-06 14:18:31 +02:00
Goutham Veeramachaneni a607873ad2 Log the stopping of TargetManager in Info
Fixes #2715

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-06 11:44:07 +02:00
Goutham Veeramachaneni 643c5837a0 Stop metrics that are 10mins ahead from now
Fixes #2893

Signed-off-by: Goutham Veeramachaneni <goutham@boomerangcommerce.com>
2017-07-04 15:34:08 +02:00
Goutham Veeramachaneni 3069bd3996 Handle scrapes with OutOfBounds metrics better
fixes #2894

Signed-off-by: Goutham Veeramachaneni <goutham@boomerangcommerce.com>
2017-07-04 11:24:13 +02:00
Fabian Reinartz 9ea748e745 Don't reallocate label set if still known
If the storage deprecates a ref, we have to re-insert with the full
label set. Typically that doesn't correlate with a new series being
created.
We can still use the allocated label set from before.
2017-06-26 14:38:57 +02:00
Fabian Reinartz 2368d2c45b retrieval: fix memory leak in scrape cache 2017-06-26 00:24:54 +02:00
Fabian Reinartz ca2b68889b Merge branch 'master' into dev-2.0 2017-06-23 13:15:44 +02:00
Brian Brazil a6ca391e6e Reject scrapes with invalid utf-8 label values. 2017-06-20 10:54:39 +01:00
Fabian Reinartz 98c2d8477a Merge pull request #2844 from Gouthamve/cobra
Move CLI commander to cobra
2017-06-19 11:59:52 +02:00
Goutham Veeramachaneni 507790a357
Rework logging to use explicitly passed logger
Mostly cleaned up the global logger use. Still some uses in discovery
package.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-16 15:52:44 +05:30
Julius Volz 6f66125809 retrieval: Fix "up" reporting for failed scrapes 2017-06-14 22:22:12 -04:00
Brian Brazil d8b4995ddd Check target labels are valid. Check for address after relabelling.
Fixes #2822
Fixes #2819
2017-06-09 16:18:19 +01:00
Fabian Reinartz 669075c6b9 Merge branch 'master' into dev-2.0 2017-06-06 09:36:51 +02:00
Fabian Reinartz eb651233ac Merge pull request #2787 from prometheus/limit2
Rework sample limit to work for 2.0
2017-06-06 08:21:12 +02:00
Chris Goller 42de0ae013 Use log.Logger interface for all discovery services 2017-06-01 11:25:55 -05:00
Brian Brazil 37bc607e96 Rework sample limit to work for 2.0
Correctly update reported series.
Increment prometheus_target_scrapes_exceeded_sample_limit_total.
Add back unittests.
Ignore stale markers when calculating sample limit.

Fixes #2770
2017-05-31 15:41:51 +01:00
Fabian Reinartz c6eed97c77 Merge pull request #2774 from prometheus/stalemem
Fix staleness memory leak
2017-05-30 15:55:37 -07:00
Fabian Reinartz bc7aff8cef retrieval: extract scrape cache 2017-05-30 09:37:23 -07:00
Brian Brazil 72a276e7ed Pass through storage errors in limitAppender. 2017-05-26 11:28:22 +01:00
Fabian Reinartz a83014f53c retrieval: fix memory leak and consumption for caches 2017-05-26 08:44:24 +02:00
Fabian Reinartz 3d8661b8d5 Add comment 2017-05-24 17:05:42 +02:00
Fabian Reinartz 43ca652217 retrieval: Don't allocate map on every scrape 2017-05-24 16:23:48 +02:00