Commit graph

121 commits

Author SHA1 Message Date
Frederic Branczyk 47538cf6ce
Merge pull request #3747 from prometheus/sched-update-throttle
Update throttle & tsdb update
2018-01-29 16:05:05 +01:00
Frederic Branczyk 73e829137b
discovery: Cleanup ticker 2018-01-29 13:51:04 +01:00
Ganesh Vernekar 66b0aa3b45 Fixed race condition in map iteration and map write in Discovery (#3735) (#3738)
* Fixed concurrent map iteration and map write in Discovery (#3735)

* discovery: Changed Lock to RLock in Collect
2018-01-28 22:24:31 +05:30
Krasi Georgiev fe926e7829 update the discover tests
the discovery test is now only testing update and get groups.
It doesn't do an e2e test but just a unit test of setting and receiving
target groups
2018-01-27 12:03:06 +00:00
Callum Styan 7dc05538f7 docs: SD implementations do not have to only send new/changed target groups (#3713) 2018-01-26 22:03:11 +00:00
Frederic Branczyk cfa0253ed8
discovery: Schedule updates to throttle 2018-01-26 16:24:44 +01:00
zemek 8a01a0fbed Set consul server default to localhost:8500 (#3703) 2018-01-24 12:14:32 +00:00
Julius Volz 09e460a647
discovery: Rename file SD mtime metric (#3723)
- "timestamp" -> "mtime" to be in line with node exporter and clearer.
- add unit suffix
2018-01-22 14:02:24 +01:00
Krasi Georgiev ec26751fd2 use mutexes for the discovery manager instead of a loop as this was a stupid idea 2018-01-17 18:12:58 +00:00
Krasi Georgiev 767faa44b6 fixed the tests
Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2018-01-15 13:39:47 +00:00
Krasi Georgiev d12e6f29fc discovery manager ApplyConfig now takes a direct ServiceDiscoveryConfig so that it can be used for the notify manager
reimplement the service discovery for the notify manager

Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2018-01-15 13:39:44 +00:00
Goutham Veeramachaneni b20a1b1b1b
Merge pull request #3654 from krasi-georgiev/discovery-handle-discoverer-updates
discovery - handle Discoverers that send only target Group updates.
2018-01-15 18:53:22 +05:30
Krasi Georgiev 790cf30fcb remove uneeded check 2018-01-15 11:52:20 +00:00
Krasi Georgiev 38938ba493 comment nits 2018-01-15 11:47:36 +00:00
Krasi Georgiev febebcd49a more comments for the future ME, and reverted the Discovery manager execution changes as these were correct in the first place 2018-01-12 22:07:21 +00:00
Krasi Georgiev 78ba5e62a6 few mote usefull comments 2018-01-12 13:58:23 +00:00
Krasi Georgiev cabce21b70 delete empty targets sets to avoid memory leaks 2018-01-12 13:10:59 +00:00
Krasi Georgiev abfd9f1920 nits 2018-01-12 12:19:52 +00:00
Shubheksha Jalan 0471e64ad1 Use shared types from the common repo (#3674)
* refactor: use shared types from common repo, remove util/config

* vendor: add common/config

* fix nit
2018-01-11 16:10:25 +01:00
Krasi Georgiev 546c29af5b return early for nil target groups 2018-01-09 16:34:23 +00:00
Callum Styan 97464236c7 comments with TargetProvider should read Discoverer instead (#3667) 2018-01-08 23:59:18 +00:00
Krasi Georgiev 77bf6bece0 discovery-manager comment update 2018-01-04 21:57:28 +00:00
Krasi Georgiev 135ea0f793 discovery manager - doesn't need sorting of the target groups so move it in the discovery manager tests as we only need it there.
discovery manager - refactor the discovery tests.

Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2018-01-04 21:41:54 +00:00
Krasi Georgiev 638818a974 some Discoverers send nil targetgroup so need to check for it when updating a group 2018-01-04 13:57:34 +00:00
Krasi Georgiev 7e28397a2c discovery - handle Discoverers that send only target Group updates rather than all Targets on every update.
Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2018-01-04 13:28:37 +00:00
Shubheksha Jalan ec94df49d4 Refactor SD configuration to remove config dependency (#3629)
* refactor: move targetGroup struct and CheckOverflow() to their own package

* refactor: move auth and security related structs to a utility package, fix import error in utility package

* refactor: Azure SD, remove SD struct from config

* refactor: DNS SD, remove SD struct from config into dns package

* refactor: ec2 SD, move SD struct from config into the ec2 package

* refactor: file SD, move SD struct from config to file discovery package

* refactor: gce, move SD struct from config to gce discovery package

* refactor: move HTTPClientConfig and URL into util/config, fix import error in httputil

* refactor: consul, move SD struct from config into consul discovery package

* refactor: marathon, move SD struct from config into marathon discovery package

* refactor: triton, move SD struct from config to triton discovery package, fix test

* refactor: zookeeper, move SD structs from config to zookeeper discovery package

* refactor: openstack, remove SD struct from config, move into openstack discovery package

* refactor: kubernetes, move SD struct from config into kubernetes discovery package

* refactor: notifier, use targetgroup package instead of config

* refactor: tests for file, marathon, triton SD - use targetgroup package instead of config.TargetGroup

* refactor: retrieval, use targetgroup package instead of config.TargetGroup

* refactor: storage, use config util package

* refactor: discovery manager, use targetgroup package instead of config.TargetGroup

* refactor: use HTTPClient and TLS config from configUtil instead of config

* refactor: tests, use targetgroup package instead of config.TargetGroup

* refactor: fix tagetgroup.Group pointers that were removed by mistake

* refactor: openstack, kubernetes: drop prefixes

* refactor: remove import aliases forced due to vscode bug

* refactor: move main SD struct out of config into discovery/config

* refactor: rename configUtil to config_util

* refactor: rename yamlUtil to yaml_config

* refactor: kubernetes, remove prefixes

* refactor: move the TargetGroup package to discovery/

* refactor: fix order of imports
2017-12-29 21:01:34 +01:00
Callum Styan d76d5de66f refactor to make timestamp collector work for multiple file_sd's 2017-12-23 10:13:11 +00:00
KalivarapuReshma a00fc883c3 Add metric for timestamp of the files file_sd is using. 2017-12-23 10:13:11 +00:00
pasquier-s 78625f85a7 Fix race condition on file SD (#3468)
The file discovery should only stop the watcher if it has been created
otherwise it may trigger a segmentation fault.
2017-12-21 10:07:43 +00:00
Krasi Georgiev 587dec9eb9 rebased and resolved conflicts with the new Discovery GUI page
Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2017-12-18 20:10:03 +00:00
Krasi Georgiev 80182a5d82 use poolKey as the pool map key to avoid multi dimensional maps 2017-12-18 17:23:47 +00:00
Krasi Georgiev 1ec76d1950 rearange the contexts variables and logic
split the groupsMerge function to set and get
other small nits
2017-12-18 17:23:47 +00:00
Krasi Georgiev f2df712166 updated README 2017-12-18 17:22:50 +00:00
Krasi Georgiev aca8f85699 fixed the tests 2017-12-18 17:22:50 +00:00
Krasi Georgiev fe6c544532 some renaming and comments fixes.
remove some select state that is most likely obsoleete and hoepfully doesn't braje anything :)
merge targets will sort by Discoverer name so we can have consistent tests for the maps.
2017-12-18 17:22:50 +00:00
Krasi Georgiev f5c2c5ff8f brake the start provider func so that can run unit tests against it. 2017-12-18 17:22:50 +00:00
Krasi Georgiev c5cb0d2910 simplify naming and API. 2017-12-18 17:22:50 +00:00
Krasi Georgiev 9c61f0e8a0 scrape pool doesn't rely on context as Stop() needs to be blocking to prevent Scrape loops trying to write to a closed TSDB storage. 2017-12-18 17:22:49 +00:00
Krasi Georgiev e405e2f1ea refactored discovery 2017-12-18 17:22:49 +00:00
Brian Brazil 81db4716c1
Mention SD moratorium in README (#3573) 2017-12-11 15:38:23 +00:00
Will Howard 6a80fc24cf Parse the normalized container.PortMappings presented by the Marathon 1.5.x API
Fixes #3465
2017-12-06 11:23:12 -05:00
Brian Brazil d7b3df5ae1 Fix staticcheck errors 2017-12-02 14:52:13 +00:00
Krasi Georgiev 29506e0bca one meaningless write to the config file to trigger anothe fsnotify (#3492) 2017-12-01 17:32:27 +00:00
Tom Wilkie 099c50ce93 Avoid empty pod UID in test. 2017-11-24 15:02:42 +00:00
Tom Wilkie 9811e90d65 Fix tests. 2017-11-24 12:24:13 +00:00
Tom Wilkie 06dc1e8797 Include Pod UID in the discovery metadata. 2017-11-20 21:09:47 +00:00
Tobias Schmidt 91be55ebf0
Merge pull request #3458 from grandbora/test-race
Fix race in test
2017-11-13 17:57:21 +01:00
Bora Tunca 493fd6bd1f Fix race in test 2017-11-13 11:47:59 -05:00
Krasi Georgiev 1005ef0a70 Fix flaky file discovery tests - sync the channel draining goroutine 2017-11-13 12:12:01 +00:00
Bora Tunca 3cc01a3088 Add more discovery tests for updating target groups (#3426)
* Adds a test covering the case where a target providers sends updated versions of the same target groups and the system should reconcile to the latest version of each of the target groups
* Refactors how input data is represented in the tests. It used to be literal declarations of necessary structs, now it is parsing yaml. Yaml declarations are half as long as the former. And these can be put in a fixture file
* Adds a tiny bit of refactoring on test timeouts
2017-11-12 03:39:08 +01:00