760 commits

Author	SHA1	Message	Date
Ayoub Mrini	af2a1cb10c	Merge pull request #15227 from aniketnk/i15185_1 ... Run discovery/kubernetes tests in parallel	2024-12-05 10:48:26 +01:00
machine424	c9f3d9b47f	doc(nomad): adjust sections about nomad_sd_config's server ... test(nomad): extend TestConfiguredService with more valid/invalid servers configs fixes https://github.com/prometheus/prometheus/issues/12306 Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-12-03 19:41:45 +01:00
hongmengning	2a1b940ae4	discovery: fix some function names in comment ... Signed-off-by: hongmengning <go@before.tech>	2024-11-25 17:33:04 +08:00
Aniket Kaulavkar	f7685caf0d	Parallelize discovery/kubernetes tests using t.Parallel() ... Signed-off-by: Aniket Kaulavkar <aniket.kaulavkar@gmail.com>	2024-11-14 10:44:03 +05:30
Matthieu MOREL	af1a19fc78	enable errorf rule from perfsprint linter ... Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2024-11-06 16:50:36 +01:00
Giedrius Statkevičius	58fedb6b61	discovery/kubernetes: optimize more gets ... Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-10-28 17:17:37 +02:00
Giedrius Statkevičius	716fd5b11f	discovery/kubernetes: use namespacedName ... Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-10-28 16:19:56 +02:00
Giedrius Statkevičius	e452308e37	discovery/kubernetes: optimize resolvePodRef ... resolvePodRef is in a hot path: ``` ROUTINE ======================== github.com/prometheus/prometheus/discovery/kubernetes.(*Endpoints).resolvePodRef in discovery/kubernetes/endpoints.go 2.50TB 2.66TB (flat, cum) 22.28% of Total . . 447:func (e *Endpoints) resolvePodRef(ref *apiv1.ObjectReference) *apiv1.Pod { . . 448: if ref == nil || ref.Kind != "Pod" { . . 449: return nil . . 450: } 2.50TB 2.50TB 451: p := &apiv1.Pod{} . . 452: p.Namespace = ref.Namespace . . 453: p.Name = ref.Name . . 454: . 156.31GB 455: obj, exists, err := e.podStore.Get(p) . . 456: if err != nil { . . 457: level.Error(e.logger).Log("msg", "resolving pod ref failed", "err", err) . . 458: return nil . . 459: } . . 460: if !exists { ``` This is some low hanging fruit that we can easily optimize. The key of an object has format "namespace/name" so generate that inside of Prometheus itself and use pooling. ``` goos: linux goarch: amd64 pkg: github.com/prometheus/prometheus/discovery/kubernetes cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz │ olddisc │ newdisc │ │ sec/op │ sec/op vs base │ ResolvePodRef-16 516.3n ± 17% 289.5n ± 7% -43.92% (p=0.000 n=10) │ olddisc │ newdisc │ │ B/op │ B/op vs base │ ResolvePodRef-16 1168.00 ± 0% 24.00 ± 0% -97.95% (p=0.000 n=10) │ olddisc │ newdisc │ │ allocs/op │ allocs/op vs base │ ResolvePodRef-16 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹ ¹ all samples are equal ``` Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2024-10-28 12:12:40 +02:00
3Juhwan	685d6d169f	refactor: reorder fields in defaultSDConfig initialization for consistency ... Signed-off-by: 3Juhwan <13selfesteem91@naver.com>	2024-10-28 10:40:49 +01:00
Ayoub Mrini	98dcd28b1a	Merge pull request #15170 from machine424/awldi ... fix(discovery): Handle cache.DeletedFinalStateUnknown in node informers' DeleteFunc	2024-10-18 17:33:08 +02:00
akunszt	08a7162502	discovery: aws/ec2 unit tests (#14364) ... * discovery: add aws/ec2 unit tests * discovery: initial skeleton for aws/ec2 unit tests This is a - very likely - not too useful unit test for the AWS SD. It is commited so other people can check the basic logic and the implementation. Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: fix linter complains about ec2_test.go Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: add basic unit test for aws This tests only the basic labelling, not including the VPC related information. Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: fix linter complains about ec2_test.go Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: other linter fixes in aws/ec2_test.go Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: implement remaining tests for aws/ec2 The coverage is not 100% but I think it is a good starting point if someone wants to improve that. Currently it covers all the AWS API calls. Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: make linter happy in aws/ec2_test.go Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: make utility funtcions private Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discover: no global variable in the aws/ec2 test Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: common body for some tests in ec2 Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: try to make golangci-lint happy Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: make every non-test function private Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: test for errors first in TestRefresh Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: move refresh tests into the function This way people can find both the test cases and the execution of the test at the same place. Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: fix copyright date Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: remove misleading comment Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: rename test for easier identification Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: use static values for the test cases Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discover: try to make the linter happy Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: drop redundant data from ec2 and use common ptr functions Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: use Error instead of Equal Signed-off-by: Arpad Kunszt <akunszt@hiya.com> * discovery: merge refreshAZIDs tests into one Signed-off-by: Arpad Kunszt <akunszt@hiya.com> --------- Signed-off-by: Arpad Kunszt <akunszt@hiya.com>	2024-10-16 14:36:37 +02:00
machine424	b1c356beea	fix(discovery): Handle cache.DeletedFinalStateUnknown in node informers' DeleteFunc ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-10-16 10:20:37 +02:00
M Viswanath Sai	16bba78f15	discovery: Improve Azure test coverage to 50% (#14586) ... * azure sd: separate refresh and refreshAzure * azure sd: create a client with mocked servers for tests * add test for refresh function --------- Signed-off-by: mviswanathsai <mviswanath.sai.met21@itbhu.ac.in>	2024-10-13 10:24:51 +02:00
Bryan Boreham	b87b88ddc2	Merge branch 'main' into consul-catalog-filter-support ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-10-08 12:20:31 +01:00
TJ Hoplock	6ebfbd2d54	chore!: adopt log/slog, remove go-kit/log ... For: #14355 This commit updates Prometheus to adopt stdlib's log/slog package in favor of go-kit/log. As part of converting to use slog, several other related changes are required to get prometheus working, including: - removed unused logging util func `RateLimit()` - forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger - move some of the json file logging functionality to use prom/common package functionality - refactored some of the new json file logging for scraping - changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers - updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition - added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2024-10-07 15:58:50 -04:00
Matthieu MOREL	ab64966e9d	fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094) ... * fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" --------- Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-10-06 16:35:29 +00:00
bas smit	73997289c3	tests: update discovery tests with new labael ... Previous commit added the pod_container_init label to discovery, so all the tests need to reflect that. Signed-off-by: bas smit <bsmit@bol.com>	2024-10-01 10:26:58 +02:00
bas smit	a10dc9298e	sd k8s: support sidecar containers in endpoint discovery ... Sidecar containers are a newish feature in k8s. They're implemented similar to init containers but actually stay running and allow you to delay startup of your application pod until the sidecar started (like init containers always do). This adds the ports of the sidecar container to the list of discovered endpoint(slice), allowing you to target those containers as well. The implementation is a copy of that of Pod discovery fixes: #14927 Signed-off-by: bas smit <bsmit@bol.com>	2024-10-01 10:26:58 +02:00
bas smit	7a90d73fa6	sd k8s: test for sidecar container support in endpoints ... This test is expected to fail, the followup will add the feature Signed-off-by: bas smit <bsmit@bol.com>	2024-10-01 10:26:58 +02:00
machine424	b5569c4070	fix(discovery): adjust how type is retrieved in Configs' MarshalYAML/UnmarshalYAML ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-09-30 12:33:07 +02:00
machine424	97f3219157	test(discovery): add a Configs test showing that the custom unmarshalling/marshalling is broken. ... This went under the radar because the utils are never called directly. We usually marshall/unmarshal Configs as embeded in a struct using UnmarshalYAMLWithInlineConfigs/MarshalYAMLWithInlineConfigs which bypasses Configs' custom UnmarshalYAML/MarshalYAML Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-09-30 12:33:07 +02:00
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868) ... * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	2024-09-10 22:32:03 +02:00
machine424	d18fa62ae9	chore(discovery): enable new-service-discovery-manager by default and drop legacymanager package ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-09-05 12:46:03 +02:00
Jan Fajerski	fe4289b502	Merge branch 'main' into HEAD	2024-09-04 18:50:00 +02:00
Jan Fajerski	00315ce15e	Merge branch 'main' into 3.0-main-sync-24-08-30 ... using -Xours Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2024-09-02 11:27:18 +02:00
machine424	d23d196db5	fix(discovery): prevent the manager from storing stale targetGroups ... Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-08-30 14:39:31 +02:00
machine424	c586c15ae6	fix(discovery): make discovery manager notify consumers of dropped targets for still defined jobs ... scrape/manager_test.go: add a test to check that the manager gets notified for targets that got dropped by discovery to reproduce: https://github.com/prometheus/prometheus/issues/12858#issuecomment-1732318102 Signed-off-by: machine424 <ayoubmrini424@gmail.com>	2024-08-28 17:39:02 +02:00
Bryan Boreham	4202be5e79	Merge branch 'release-2.54' into merge-2.54.1-into-main	2024-08-27 12:04:48 +01:00
beorn7	0f760f63dd	lint: Revamp our linting rules, mostly around doc comments ... Several things done here: - Set `max-issues-per-linter` to 0 so that we actually see all linter warnings and not just 50 per linter. (As we also set `max-same-issues` to 0, I assume this was the intention from the beginning.) - Stop using the golangci-lint default excludes (by setting `exclude-use-default: false`. Those are too generous and don't match our style conventions. (I have re-added some of the excludes explicitly in this commit. See below.) - Re-add the `errcheck` exclusion we have used so far via the defaults. - Exclude the signature requirement `govet` has for `Seek` methods because we use non-standard `Seek` methods a lot. (But we keep other requirements, while the default excludes completely disabled the check for common method segnatures.) - Exclude warnings about missing doc comments on exported symbols. (We used to be pretty adamant about doc comments, but stopped that at some point in the past. By now, we have about 500 missing doc comments. We may consider reintroducing this check, but that's outside of the scope of this commit. The default excludes of golangci-lint essentially ignore doc comments completely.) - By stop using the default excludes, we now get warnings back on malformed doc comments. That's the most impactful change in this commit. It does not enforce doc comments (again), but _if_ there is a doc comment, it has to have the recommended form. (Most of the changes in this commit are fixing this form.) - Improve wording/spelling of some comments in .golangci.yml, and remove an outdated comment. - Leave `package-comments` inactive, but add a TODO asking if we should change that. - Add a new sub-linter `comment-spacings` (and fix corresponding comments), which avoids missing spaces after the leading `//`. Signed-off-by: beorn7 <beorn@grafana.com>	2024-08-22 17:36:11 +02:00
Jan Fajerski	5138922b0d	Merge branch 'main' into 3.0-main-sync-24-08-21	2024-08-21 09:09:36 +02:00
ouyang1204@gmail.com	89dee48cc8	fix the issue of failing to match the first network when the container is reconnected to a new network ... Signed-off-by: ouyang1204@gmail.com <ouyang1204@gmail.com>	2024-08-19 21:26:25 +08:00
Arve Knudsen	3a78e76282	Upgrade golangci-lint to v1.60.1 ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-08-18 12:13:25 +02:00
Julien Pivotto	7711cd5ab5	Remove deprecated storage.tsdb.retention flag ... Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> Signed-off-by: Julien <roidelapluie@o11y.eu>	2024-08-16 13:53:09 +02:00
cuiweiyuan	1800af54f0	chore: fix some function names ... Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>	2024-08-15 13:57:21 +08:00
Björn Rabenstein	3f16a2e7de	Merge pull request #14543 from jan--f/3.0-main-sync-24-08-01 ... 3.0 main sync 24 08 01	2024-08-13 15:54:13 +02:00
Julien	3933cba052	Merge pull request #14365 from simonpasquier/fix-12884 ... discovery(k8s): remove support for API versions no longer served	2024-08-09 12:48:54 +02:00
Bryan Boreham	79a0ba9d64	Merge pull request #13503 from tylitianrui/chore/remove_redundance ... remove redundant code	2024-07-30 12:44:03 +01:00
Bryan Boreham	ce3bd4abea	Update for Docker deprecation ... Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2024-07-17 17:03:32 +01:00
Simon Pasquier	145988d48f	discovery(k8s): remove support for API versions no longer served ... This commit removes support for the following API versions: * `discovery.k8s.io/v1beta1` API version of EndpointSlice (no longer served as of v1.25). * `networking.k8s.io/v1beta1` API version of Ingress (no longer served as of v1.22). Closes #12884 Signed-off-by: Simon Pasquier <spasquie@redhat.com>	2024-07-04 14:54:27 +02:00
Paulo Dias	f4b1fcb73e	discovery: add support for gathering flavor name in Openstack discovery (#14312) ... * feat: add support for gathering flavor name in Openstack discovery Signed-off-by: Paulo Dias <paulodias.gm@gmail.com> * Update instance.go Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Paulo Dias <44772900+paulojmdias@users.noreply.github.com> * Update configuration.md Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Paulo Dias <44772900+paulojmdias@users.noreply.github.com> * fix: fix linting Signed-off-by: Paulo Dias <paulodias.gm@gmail.com> * fix: fix instance type Signed-off-by: Paulo Dias <paulodias.gm@gmail.com> * Update docs/configuration/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Paulo Dias <44772900+paulojmdias@users.noreply.github.com> --------- Signed-off-by: Paulo Dias <paulodias.gm@gmail.com> Signed-off-by: Paulo Dias <44772900+paulojmdias@users.noreply.github.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Co-authored-by: Simon Pasquier <spasquie@redhat.com>	2024-06-30 19:18:18 +02:00
Bryan Boreham	c5040c5ea9	Merge pull request #10490 from DrAuYueng/fix-docker-sd-service-missing ... [ENHANCEMENT] Docker SD: add MatchFirstNetwork for containers with multiple networks Fixes docker sd service misssing in shared mode and deduplicate targets by network	2024-06-26 12:33:50 +01:00
Arve Knudsen	d902116b41	Fix various linting errors ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2024-06-24 16:11:53 -07:00
unknown	0d25931049	rebase main and adjust the configuration ... Signed-off-by: ouyang1204@gmail.com <ouyang1204@gmail.com>	2024-06-21 19:10:18 +08:00
akunszt	2aaf99dd0a	discovery: aws: expose Primary IPv6 addresses as label, partially fixes #7406 (#14156) ... * discovery: aws: expose Primary IPv6 addresses as label Add __meta_ec2_primary_ipv6_addresses label. This label contains the Primary IPv6 address for every ENI attached to the EC2 instance. It is ordered by the DeviceIndex and the missing elements (interface without Primary IPv6 address) are kept in the list. --------- Signed-off-by: Arpad Kunszt <akunszt@hiya.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>	2024-06-20 14:36:20 +01:00
machine424	94d28cd6cf	chore(notifier): add a reproducer for https://github.com/prometheus/prometheus/issues/13676 ... to show "targets groups update" starvation when the notifications queue is full and an Alertmanager is down. The existing `TestHangingNotifier` that was added in https://github.com/prometheus/prometheus/pull/10948 doesn't really reflect the reality as the SD changes are manually fed into `syncCh` in a continuous way, whereas in reality, updates are only resent every `updatert`. The test added here sets up an SD manager and links it to the notifier. The SD changes will be triggered by that manager as it's done in reality. Signed-off-by: machine424 <ayoubmrini424@gmail.com> Co-authored-by: Ethan Hunter <ehunter@hudson-trading.com>	2024-06-19 09:43:52 +02:00
György Krajcsovits	b18e2d3ef6	Format linter pragma correctly ... Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-06-05 16:50:09 +02:00
György Krajcsovits	180059e6fa	Update scaleway-sdk-go and silence deprecation warnings ... This reverts commit 777daea86e.	2024-06-05 16:37:58 +02:00
György Krajcsovits	0ee4cf8d94	Update Go dependencies for 2.53 ... Ran "make update-all-go-deps" , but do not bump go version and remove toolchain clause from go.mod Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2024-06-04 18:02:21 +02:00
Arthur Silva Sens	d73537ccc0	Merge branch 'main' into resolveconflicts-2.52.1	2024-05-31 09:36:17 -03:00
David Andruczyk	851f68d1cc	BUGFIX: Need seperate listOptions structs since linodego writes into them for pagination ... Signed-off-by: David Andruczyk <dandrucz@akamai.com>	2024-05-27 17:19:58 +00:00