From 1a7d82fb69d79214b156dddf75dcfe62df268243 Mon Sep 17 00:00:00 2001 From: Julius Volz Date: Sat, 13 Aug 2022 11:27:56 +0200 Subject: [PATCH] Remove CircleCI Windows tests as they are too flaky Windows tests on CircleCI are constantly failing for a variety of reasons in a flaky way, putting pain on anyone submitting PRs. Nobody seems to be addressing this at the moment, we just ignore them or retry them until they succeed, defeating their purpose. So I suggest removing the Windows tests completely until someone wants to figure out why they are broken and fix them. For reference, here's only some of the observed failures: ``` === RUN TestUpdateMissedEvalMetrics manager_test.go:1325: Error Trace: C:\go\src\github.com\prometheus\prometheus\rules\manager_test.go:1325 C:\go\src\github.com\prometheus\prometheus\rules\manager_test.go:1329 Error: Not equal: expected: 2 actual : 1 Test: TestUpdateMissedEvalMetrics Signed-off-by: Julius Volz --- FAIL: TestUpdateMissedEvalMetrics (7.60s) ``` ``` === RUN TestQueryStatistics/sum_by_(b)_(max_over_time(metricWith3SampleEvery10Seconds[60s]_@_30)) engine_test.go:1217: Error Trace: C:\go\src\github.com\prometheus\prometheus\promql\engine_test.go:1217 Error: Not equal: expected: 8 actual : 7 Test: TestQueryStatistics/sum_by_(b)_(max_over_time(metricWith3SampleEvery10Seconds[60s]_@_30)) Messages: Peak samples mismatch --- FAIL: TestQueryStatistics/sum_by_(b)_(max_over_time(metricWith3SampleEvery10Seconds[60s]_@_30)) (0.00s) ``` ``` === RUN TestNewHTTPCACert target_test.go:240: Get "https://127.0.0.1:50272": x509: certificate signed by unknown authority (possibly because of "x509: cannot verify signature: insecure algorithm SHA1-RSA (temporarily override with GODEBUG=x509sha1=1)" while trying to verify candidate authority certificate "Prometheus Test CA") 2022/08/12 15:10:23 http: TLS handshake error from 127.0.0.1:50273: remote error: tls: bad certificate --- FAIL: TestNewHTTPCACert (0.02s) ``` ``` Too long with no output (exceeded 10m0s): context deadline exceeded ``` Signed-off-by: Julius Volz --- .circleci/config.yml | 25 ------------------------- 1 file changed, 25 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index d6fd08755..e024aaebb 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -71,27 +71,6 @@ jobs: paths: - ~/.npm - test_windows: - executor: - name: win/default - shell: powershell - working_directory: /go/src/github.com/prometheus/prometheus - steps: - - checkout - - run: - # Temporary workaround until circleci updates go. - command: | - choco upgrade -y golang - - run: - command: refreshenv - - run: - command: | - $TestTargets = go list ./... | Where-Object { $_ -NotMatch "(github.com/prometheus/prometheus/discovery.*|github.com/prometheus/prometheus/config|github.com/prometheus/prometheus/web)"} - go test $TestTargets -vet=off -v - environment: - GOGC: "20" - GOOPTS: "-p 2" - test_golang_oldest: executor: golang_oldest steps: @@ -139,10 +118,6 @@ workflows: filters: tags: only: /.*/ - - test_windows: - filters: - tags: - only: /.*/ - prometheus/build: name: build parallelism: 3