Commit graph

434 commits

Author SHA1 Message Date
Anders Daljord Morken 8633ac180e Strip stray whitespace from bearer token file
Apart from not trying to send a newline in a HTTP header,
this also allows Prometheus to build and pass tests with Go 1.7,
which features stricter checking of HTTP headers.
2016-08-17 15:36:18 +02:00
Frederic Branczyk 7714b9c781 move relabeling functionality to its own package
also remove the returned error as it was always nil
2016-08-09 14:19:20 +02:00
Jimmi Dyson 6c8080607f
Kubernetes SD: Add node name and host IP to pod discovery 2016-07-20 12:00:54 +01:00
Dmitry Vorobev 273e457da4 web: return status code and error message for config resource 2016-07-15 10:15:24 +02:00
beorn7 064b57858e Consistently use the Seconds() method for conversion of durations
This also fixes one remaining case of recording integral numbers
of seconds only for a metric, i.e. this will probably fix #1796.
2016-07-07 15:24:35 +02:00
Fabian Reinartz 4591a2623b discovery/kubernetes: filter pod/container, service/endpoint
This change distinguishes and filters by pod/container and
service/endpoint in the respective sub-SDs.
2016-07-05 14:24:17 +02:00
Fabian Reinartz 0ff354341b discovery/kubernetes: remove unused channel 2016-07-05 14:22:12 +02:00
Fabian Reinartz 7221228843 discovery/kubernetes: select between discovery role
This adds `role` field to the Kubernetes SD config, which indicates
which type of Kubernetes SD should be run.
This no longer allows discovering pods and nodes with the same SD
configuration for example.
2016-07-05 14:22:12 +02:00
Fabian Reinartz e0f8caacd7 discovery/kubernetes: extract service endpoint discovery
This extract discovery of services and their endpoints into its own
type.
2016-07-05 10:26:23 +02:00
Fabian Reinartz fdbe28df85 discovery/kubernetes: extract node discovery
This change extracts node discovery into its own type.
2016-07-01 19:31:04 +02:00
Fabian Reinartz 8a97c211a8 discovery/kubernetes: extract pod discovery
This change extracts pod discovery into its own type.
2016-07-01 19:30:00 +02:00
Fabian Reinartz e03e138d34 discovery: consolidate constructors into single file 2016-07-01 19:30:00 +02:00
Fabian Reinartz 57333d1831 discovery/kubernetes: add missing locking 2016-07-01 17:07:13 +02:00
Fabian Reinartz 44036a08d0 Merge pull request #1725 from nicholascapo/use-consul-service-address
discovery: use consul service address if available
2016-06-30 09:30:14 +02:00
Nicholas Capo 84334a8410 discovery: use consul service address if available 2016-06-15 19:27:05 -05:00
Fabian Reinartz 4aeab798e8 Merge pull request #1738 from prometheus/release-0.19
Forward-merge 0.19 fixes into master
2016-06-14 18:11:47 +02:00
Fabian Reinartz 3c80609fce Merge pull request #1737 from prometheus/fabxc-0.19.3
Bump version to 0.19.3
2016-06-14 18:04:56 +02:00
Fabian Reinartz 4c864c8a88 retrieval: don't sync to uninitialized scrape pool
This change does just signal a scrape target update to the scraping loop
once an initial target set is fetched.
Before, the scrape pool was directly synced, causing a race against an
uninitialized scrape pool.

Fixes #1703
2016-06-14 14:18:40 +02:00
Fabian Reinartz d0eeae9d0e retrieval: don't sync to uninitialized scrape pool
This change does just signal a scrape target update to the scraping loop
once an initial target set is fetched.
Before, the scrape pool was directly synced, causing a race against an
uninitialized scrape pool.

Fixes #1703
2016-06-14 14:04:22 +02:00
beorn7 03adbe57e4 discovery/marathon: Fix race conditions in test
The concurrency applied before is in most cases not even needed. With
a cap=1 channel, most tests are much cleaner.

TestMarathonSDRunAndStop was trickier. It could even have blocked
before.

This also includes a general refactoring of the whole file.
2016-06-14 13:12:53 +02:00
Björn Rabenstein 2ea3a837c3 Merge pull request #1731 from prometheus/release-0.19
Forward-merge 0.19 fixes to master
2016-06-14 09:21:52 +02:00
rohit01 47dd5f74ba discovery/marathon: #1722 - ignore apps with zero ports 2016-06-14 04:44:00 +05:30
Fabian Reinartz 0f21bd31ca config: deprecate target_groups for static_configs
This change deprecates the `target_groups` option in favor
of `static_configs`. The old configuration is still accepted
but prints a warning.
Configuration loading errors if both options are set.
2016-06-08 15:55:25 +02:00
Brian Brazil 05b918a024 Merge pull request #1713 from mattbostock/document_drop
Relabel: Document whole label set is dropped
2016-06-07 17:22:34 +01:00
Matt Bostock 329a00e44f Relabel: Document whole label set is dropped
From the documentation and current tests, it wasn't immediately clear to
me whether the `target` being dropped as the result of a 'drop' action
was a label key-value pair or the entire labelset.

Add a test that documents this behaviour.

Documentation: https://prometheus.io/docs/operating/configuration/
2016-06-07 17:13:30 +01:00
Jimmi Dyson 206bcfcdaa
Kubernetes SD: Remove kubeletPort config option 2016-06-07 12:34:55 +01:00
Jimmi Dyson d48297c904
Kubernetes SD: Add labels for all node addresses and discover node port if available 2016-06-07 12:34:49 +01:00
Fabian Reinartz 26b1c89469 Merge pull request #1702 from pdbogen/master
Initial local pods before using the pod list to initialize pod targets
2016-06-06 15:14:49 +02:00
Patrick Bogen 1e6770cdc5 Initial local pods before using the pod list to initialize pod targets; include more logging in pod target creation 2016-06-02 17:49:21 -07:00
Ali Reza c81b4e8a87 change config names to files for consistency 2016-05-30 07:47:58 +07:00
Gregory G. Tseng 4ceedffe86 Unexport testing constant 2016-05-26 14:42:17 -07:00
Gregory G. Tseng 7997c14b0d Add ServerName into TLS Config 2016-05-26 14:24:49 -07:00
Fabian Reinartz 74c448386c Merge pull request #1665 from prometheus/fabxc-retrpanic
Fix kubernetes SD crash
2016-05-25 17:13:27 -07:00
Fabian Reinartz 12b03db373 retrieval: handle nil target groups from updates 2016-05-25 16:59:16 -07:00
Fabian Reinartz ea36efbbd1 retrieval: document panic behavior 2016-05-25 16:17:25 -07:00
Fabian Reinartz a5ba166935 retrieval: don't panic on non-HTTP scheme 2016-05-25 16:05:20 -07:00
Tobias Schmidt 0c6ed9d437 Fix type usage in Kubernetes discovery
The event types have been recently changed to be exported types which
wasn't reflected in the new pod discovery.
2016-05-20 11:03:19 -04:00
Fabian Reinartz f7ed2ff706 Merge pull request #1644 from prometheus/beorn7/logging
Add missing logging of out-of-order samples
2016-05-20 05:52:00 -07:00
Fabian Reinartz dec56838fc Merge pull request #1449 from pdbogen/master
k8s pod discovery
2016-05-20 05:45:09 -07:00
Patrick Bogen 89940eb48d Write tests to include testing determinancy of various slice orders; ensure that container order is deterministic 2016-05-19 10:57:23 -07:00
Patrick Bogen b3350d872a Add one label named for each port name, mapping it to port number; add corresponding tests; prefix port list label with a comma 2016-05-19 10:37:11 -07:00
beorn7 d43c0159aa Fix style issues in retrieval/... 2016-05-19 17:14:04 +02:00
beorn7 45e5775f9b Add missing logging of out-of-order samples
So far, out-of-order samples during rule evaluation were not logged,
and neither scrape health samples. The latter are unlikely to cause
any errors. That's why I'm logging them always now. (It's alway highly
irregular should it happen.) For rules, I have used the same plumbing
as for samples, just with a different wording in the message to mark
them as a result of rule evaluation.
2016-05-19 16:22:53 +02:00
Patrick Bogen ae413704e8 kubernetes pod-level discovery 2016-05-18 17:18:52 -07:00
Fabian Reinartz 2ca9ee7b0d Merge pull request #1612 from prometheus/stn-dns-mv
[discovery][dns] move dns to own package
2016-05-06 12:23:59 +02:00
stuart nelson d959d2b90a discovery/dns: Maintain argument order consistency 2016-05-06 11:14:26 +02:00
stuart nelson d02591814b discovery/dns: move dns to own package 2016-05-06 11:14:26 +02:00
Shawn Smith d7481f266e fix typo 2016-05-04 13:42:27 +09:00
Tobias Schmidt 3ce895e86d Merge pull request #1608 from prometheus/fix-printf
Fix format argument in retrieval test.
2016-05-01 17:44:16 -04:00
Julius Volz 97b018d26d Fix format argument in retrieval test. 2016-05-01 23:37:45 +02:00
Fabian Reinartz f94fc76608 Merge pull request #1592 from prometheus/fabxc-consul-ref
discovery: sanitize Consul service discovery
2016-04-30 21:18:33 +02:00
Fabian Reinartz 289f306dd9 Merge pull request #1590 from prometheus/fabxc-marathon-ref
Fix basic issues in marathon SD
2016-04-30 21:17:22 +02:00
Fabian Reinartz 76076bfb47 discovery: simplify client initialization 2016-04-30 21:07:49 +02:00
Fabian Reinartz b5bfb502df discovery: properly check context on chan send 2016-04-30 11:57:20 +02:00
Fabian Reinartz 9f8feb9ff6 discovery: consolidate Marathon SD files 2016-04-30 11:56:11 +02:00
Fabian Reinartz 086f7caceb discovery: extract Consul shouldWatch logic 2016-04-30 11:50:19 +02:00
Fabian Reinartz e805e68c01 discovery: sanitize Consul service discovery
This commits simplifies the SD's structure and ensures that all
channel sends are checked against a canceled context.
2016-04-30 11:50:19 +02:00
Fabian Reinartz 5837e6a97f discovery: move consul SD into own package 2016-04-25 16:56:27 +02:00
beorn7 d566808d40 Bring back logging of discarded samples
But only on DEBUG level.

Also, count and report the two cases of out-of-order timestamps on the
one hand and same timestamp but different value on the other hand
separately.
2016-04-25 16:43:52 +02:00
Fabian Reinartz 585ab6b163 Merge pull request #1494 from iamseth/master
Add discovery capability for Microsoft Azure
2016-04-21 13:49:44 +02:00
Jonathan Boulle 38098f8c95 Add missing license headers
Prometheus is Apache 2 licensed, and most source files have the
appropriate copyright license header, but some were missing it without
apparent reason. Correct that by adding it.
2016-04-13 16:08:22 +02:00
Seth Miller 0988e3b937 Add support for Azure discovery
This change adds the ability to do target discovery with Microsoft's Azure platform.
2016-04-06 22:47:02 -05:00
Fabian Reinartz 769389e559 Fix potential race in ctx intialization 2016-04-05 20:27:31 +02:00
Tobias Schmidt e82ef154ee Remove unused code leftovers 2016-04-02 20:20:55 -04:00
stuart nelson dbe5d18b6e Instrument scrape pool sync()
Instruments:
- duration
- count
2016-03-14 18:30:16 +01:00
stuart nelson 813f61e551 Merge pull request #1484 from prometheus/instrument-retrieval
Instrument retrieval/scrape.go
2016-03-11 12:26:00 +01:00
stuart nelson a1ee77601a Instrument the duration of the reload function 2016-03-11 12:12:42 +01:00
Fabian Reinartz 895f2f092f Fix flaky scrape test
t
2016-03-09 16:00:33 +01:00
Fabian Reinartz f2e359962c Sort exported targets 2016-03-08 17:12:27 +01:00
Fabian Reinartz 56fc9bdff3 Handle closed target provider channel
This fixes the case where a target provider closes the update
channel and exits before the context is canceled.
This should only be true for the static provider but it's safer
to generally handle this case.
2016-03-08 15:49:03 +01:00
beorn7 d44b83690e Fix flaky file-sd test 2016-03-07 15:39:18 +01:00
Fabian Reinartz ddc74f712b Add sortable target list 2016-03-02 09:10:20 +01:00
Fabian Reinartz 499f4af4aa Test target URL 2016-03-01 14:49:57 +01:00
Fabian Reinartz 50c2f20756 Add targetScraper tests 2016-03-01 14:33:28 +01:00
Fabian Reinartz 1ede7b9d72 Consolidate TargetStatus into Target.
This commit simplifies the TargetHealth type and moves the target
status into the target itself. This also removes a race where error
and last scrape time could have been out of sync.
2016-03-01 14:33:21 +01:00
Fabian Reinartz 2060a0a15b Turn target group members into plain lists.
As the scrape pool deduplicates targets now, it is no longer necessary
to store a hash map for members of each group.
2016-03-01 14:33:12 +01:00
Fabian Reinartz 0d7105abee Remove scrape config from Target.
This commit removes the scrapeConfig entirely from Target.
All identity defining parameters are thus immutable now and the mutex
can be removed..

Target identity is now correctly defined by the labels and the full URL.
This in particular includes URL parameters that are not specified in the
label set.

Fingerprint is also removed from hash to remove an unnecessary tight coupling
to the common/model package.
2016-03-01 14:32:57 +01:00
Fabian Reinartz 75681b691a Extract HTTP client from Target.
The HTTP client is the same across all targets with the same
scrape configuration. Thus, this commit moves it into the scrape
pool.
2016-03-01 14:31:57 +01:00
Fabian Reinartz 9bea27ae8a Add scraping tests 2016-03-01 14:00:48 +01:00
Fabian Reinartz 76a8c6160d Deduplicate targets in scrape pool.
With this commit the scrape pool deduplicates incoming
targets before scraping them. This way multiple target providers
can produce the same target but it will be scraped only once.
2016-03-01 13:50:51 +01:00
Fabian Reinartz 84f74b9a84 Apply new scrape config on reload.
This commit updates a target set's scrape configuration
on reload. This will cause all running scrape loops to be
stopped and started again with new parameters.
2016-03-01 13:50:51 +01:00
Fabian Reinartz 02f635dc24 Remove interval/timeout from Target internals 2016-03-01 13:50:51 +01:00
Fabian Reinartz 775316f8d2 Move appender construction from Target to scrapePool 2016-03-01 13:50:51 +01:00
Fabian Reinartz fbe251c2df Fix scrape interval length calculation 2016-03-01 13:48:36 +01:00
Fabian Reinartz 1a3253e8ed Make scrape time unambigious.
This commit changes the scraper interface to accept a timestamp
so the reported timestamp by the caller and the timestamp
attached to samples does not differ.
2016-03-01 13:48:36 +01:00
Fabian Reinartz 2bb8ef99d1 Test scrape loop behavior. 2016-03-01 13:48:36 +01:00
Fabian Reinartz c7bbe95597 Remove outdated target tests 2016-03-01 13:48:36 +01:00
Fabian Reinartz 05de8b7f8d Extract target scraping into scrape loop.
This commit factors out the scrape loop handling into
its own data structure.
For the transition it will be directly attached to the
target.
2016-03-01 13:48:36 +01:00
Fabian Reinartz cebba3efbb Simplify and fix TargetManager reloading 2016-03-01 13:48:36 +01:00
Fabian Reinartz da99366f85 Consolidate Target.Update into constructor.
The Target.Update method is no longer needed.
2016-03-01 13:48:36 +01:00
Fabian Reinartz d15adfc917 Preserve target state across reloads.
This commit moves Scraper handling into a separate scrapePool type.
TargetSets only manage TargetProvider lifecycles and sync the
retrieved updates to the scrapePool.

TargetProviders are now expected to send a full initial target set
within 5 seconds. The scrapePools preserve target state across reloads
and only drop targets after the initial set was synced.
2016-03-01 13:48:36 +01:00
Fabian Reinartz 5b30bdb610 Change TargetProvider interface.
This commit changes the TargetProvider interface to use a
context.Context and send lists of TargetGroups, rather than
single ones.
2016-03-01 13:48:36 +01:00
Fabian Reinartz bb6dc3ff78 Remove old tests 2016-03-01 13:48:36 +01:00
Fabian Reinartz 5bfa4cdd46 Simplify target update handling.
We group providers by their scrape configuration. Each provider produces
target groups with an unique identifier.

On stopping a set of target providers we cancel the target providers,
stop scraping the targets and wait for the scrapers to finish.

On configuration reload all provider sets are stopped and new ones
are created. This will make targets disappear briefly on configuration
reload. Potentially scrapes are missed but due to the consistent
scrape intervals implemented recently, the impact is minor.
2016-03-01 13:48:36 +01:00
Jimmi Dyson e59b7c15a3 Kubernetes SD: Fix node IP discovery 2016-03-01 12:24:52 +00:00
beorn7 33a50e69f7 Fix a deadlock
Double acquisition of the RLock usually doesn't blow up, but if the
write lock is called for between the two RLock's, we are deadlocked.

This deadlock does not exist in release-0.17, BTW.
2016-02-29 16:34:29 +01:00
beorn7 fd5108b038 Fix a targetmanager test 2016-02-22 16:43:48 +01:00
Fabian Reinartz 6df1f49c13 Remove fullLabels method and fix target updating
With recent changes to a Target's internal data representation
updating by fullLabels() assigns the additional default
instance label. This breaks target identity comparison and causes
identical targets from service discovery to be constantly swapped.
2016-02-22 13:06:30 +01:00
Fabian Reinartz 825831e98f Use fingerprint for target identity comparison
So far we were using the InstanceIdentifier to compare equality of targets.
This is not always accurate, for example for the blackbox exporter where the 
actual target is in the parameter.
2016-02-17 16:34:53 +01:00
Fabian Reinartz 66767121ab Handle scrape timeout on request.
For historic reasons we were enforcing a timeout directly
via the TCP dialer. This is no longer necessary for quite a while now.
Switching to context.Context will allow us to properly terminate
requests on shutdown as well.
2016-02-16 11:46:02 +01:00