flake: Test_localSub_init/CrossSubjectListenerIsolation

@sreya

CI Run Link: https://github.com/coder/coder/actions/runs/26576090091
Failing Job: https://github.com/coder/coder/actions/runs/26576090091/job/78296339738 (test-go-pg-17)
Failure time: 2026-05-28T13:08:36Z (same window as Slack alert)

Commit Info:

SHA: 12520ee964270e561e0e1b75b765a4194c159c4a
Author: Danny Kopping
Commit: coder/coder@12520ee

Failure evidence:

=== FAIL: coderd/x/nats Test_localSub_init/CrossSubjectListenerIsolation (25.04s)
    pubsub_internal_test.go:321:
        Error Trace: /home/runner/work/coder/coder/coderd/x/nats/pubsub_internal_test.go:321
        Error:       Condition never satisfied
        Test:        Test_localSub_init/CrossSubjectListenerIsolation

Error analysis:

The failing assertion is the require.Eventually that expects fastCount >= total in Test_localSub_init/CrossSubjectListenerIsolation.
This indicates the fast subscriber did not receive all expected messages within testutil.WaitLong when a slow subscriber is blocked, despite the test expecting isolation.

Root cause classification:

Likely flaky test / timing/backpressure in the NATS local pubsub implementation when a slow subscriber blocks and a fast subscriber should still drain.

Data race / panic / OOM checks:

No WARNING: DATA RACE, panic:, or OOM indicators observed in the job logs around the failure.

Precise assignment analysis:

Intended blame command:
- git blame -L 316,333 coderd/x/nats/pubsub_internal_test.go
Recent file history:
- git log --oneline -10 --follow coderd/x/nats/pubsub_internal_test.go
- Most recent substantive change: f6f284e “feat: add initial NATS implementation (#25602)” by Jon Ayers.
Jon Ayers corresponds to GitHub user @sreya per team mapping.

Assignment:

Best candidate: @sreya (owner of initial NATS implementation; please reassign if ownership differs).

Related issues search (coder/internal):

"Test_localSub_init"
"CrossSubjectListenerIsolation"
"pubsub_internal_test.go" / "coderd/x/nats"
"Condition never satisfied"
No matches found.

Reproduction (best effort):

go test ./coderd/x/nats -run Test_localSub_init/CrossSubjectListenerIsolation -count=1

New occurrence in main CI:

CI run: https://github.com/coder/coder/actions/runs/26582325717
Failing job: https://github.com/coder/coder/actions/runs/26582325717/job/78320555741 (test-go-pg ubuntu-latest)
Failure time: 2026-05-28T15:07:26Z (same day as Slack alert)

Commit info:

SHA: 6df1536256475730d2fdba72083d3142c3089a25
Author: Cian Johnston
Commit: coder/coder@6df1536

Failure evidence:

=== FAIL: coderd/x/nats Test_localSub_init/CrossSubjectListenerIsolation (25.06s)
    pubsub_internal_test.go:321:
        Error Trace: /home/runner/work/coder/coder/coderd/x/nats/pubsub_internal_test.go:321
        Error:       Condition never satisfied
        Test:        Test_localSub_init/CrossSubjectListenerIsolation

No data race / panic / OOM indicators observed in the job logs around the failure.

New occurrence on main:

CI Run Link: https://github.com/coder/coder/actions/runs/26586696037
Failing Job: https://github.com/coder/coder/actions/runs/26586696037/job/78334548165 (test-go-pg-17, run_attempt 1)
Commit: coder/coder@094fe97 (DevCats) - "chore(aibridge): add AWS PRM user-agent attribution for Bedrock calls (#25221)"

Failure evidence:

=== FAIL: coderd/x/nats Test_localSub_init/CrossSubjectListenerIsolation (25.05s)
    pubsub_internal_test.go:321:
        Error: Condition never satisfied

Notes:

No panic/OOM/data race indicators found in the retrieved logs.

New occurrence:

CI Run: https://github.com/coder/coder/actions/runs/26714105970
Failing Job: https://github.com/coder/coder/actions/runs/26714105970/job/78729525296 (test-go-pg (ubuntu-latest))
Failure time: 2026-05-31T13:43:20Z (same window as Slack alert)

Commit Info:

SHA: 9c111a2be2ed87490083a1224e86f04cb7bcc6ba
Author: Dean Sheather
Commit: coder/coder@9c111a2

Failure evidence:

=== FAIL: coderd/x/nats Test_localSub_init/CrossSubjectListenerIsolation (25.05s)
    pubsub_internal_test.go:316:
        Error Trace: /home/runner/work/coder/coder/coderd/x/nats/pubsub_internal_test.go:316
        Error:       Condition never satisfied
        Test:        Test_localSub_init/CrossSubjectListenerIsolation

Notes:

Same failure signature as before (fast subscriber did not reach total within testutil.WaitLong).
No data race warnings, panic traces, or OOM indicators found in the job logs around the failure.

New occurrence:

CI Run: https://github.com/coder/coder/actions/runs/26747280992
Job: https://github.com/coder/coder/actions/runs/26747280992/job/78825984726 (test-go-pg-17)
Failure time: 2026-06-01T09:49:59Z
Commit: 8b7e040105c10e9a5bad5689da607c1f32f9c9e7 (Mathias Fredriksson) coder/coder@8b7e040

Failure evidence:

=== FAIL: coderd/x/nats Test_localSub_init/CrossSubjectListenerIsolation (25.04s)
    pubsub_internal_test.go:316:
        Error Trace: /home/runner/work/coder/coder/coderd/x/nats/pubsub_internal_test.go:316
        Error:       Condition never satisfied
        Test:        Test_localSub_init/CrossSubjectListenerIsolation

Notes:

No panic, OOM, or data race warnings observed in this run’s logs around the failure.
Most recent substantive change in this test came from 5cdc9e2 “feat: add nats cluster peer support (#25632)” (Jon Ayers), which refactored this test and nearby helpers.

New occurrence in main:

CI run: https://github.com/coder/coder/actions/runs/26825987931
Job: https://github.com/coder/coder/actions/runs/26825987931/job/79095939566 (test-go-pg-17)
Failure time: 2026-06-02T14:39:45Z

Commit:

5088b5fa5fd8fafd080eddef90d46c659dfd6e53 (Ethan)
coder/coder@5088b5f

Failure evidence:

=== FAIL: coderd/x/nats Test_localSub_init/CrossSubjectListenerIsolation (25.05s)
    pubsub_internal_test.go:316:
        Error: Condition never satisfied

Root cause signal: flaky timing/backpressure in local NATS pubsub test (fast subscriber did not satisfy eventually condition).

New occurrence on main:

CI Run: https://github.com/coder/coder/actions/runs/26826917406
Failed Job: https://github.com/coder/coder/actions/runs/26826917406/job/79099249751 (test-go-pg (ubuntu-latest))
Failure time: 2026-06-02T14:55:12Z (same window as Slack alert)

Commit:

SHA: 170c33a4755a900c9b12659eed2c46db99e3062a
Author: Zach
Commit: coder/coder@170c33a

Failure evidence:

=== FAIL: coderd/x/nats Test_localSub_init/CrossSubjectListenerIsolation (25.04s)
    pubsub_internal_test.go:316:
        Error Trace: /home/runner/work/coder/coder/coderd/x/nats/pubsub_internal_test.go:316
        Error:       Condition never satisfied
        Test:        Test_localSub_init/CrossSubjectListenerIsolation

Notes:

Same failure signature as prior occurrences (Condition never satisfied in CrossSubjectListenerIsolation).
No data race, panic, or OOM indicators observed in the job logs around the failure.

New occurrence:

CI run: https://github.com/coder/coder/actions/runs/26908078930
Failing job: https://github.com/coder/coder/actions/runs/26908078930/job/79379633700 (test-go-pg ubuntu-latest)
Failure time: 2026-06-03T19:44:51Z
Commit: coder/coder@5b692bf (Spike Curtis)

Failure evidence:

=== FAIL: coderd/x/nats Test_localSub_init/CrossSubjectListenerIsolation (25.04s)
    pubsub_internal_test.go:316:
        Error:       Condition never satisfied

No WARNING: DATA RACE, panic, or OOM indicators observed in the job logs around the failure.

New occurrence in test-go-pg (ubuntu-latest).

CI Run: https://github.com/coder/coder/actions/runs/26942567262
Failing job: https://github.com/coder/coder/actions/runs/26942567262/job/79487446261
Failure time: 2026-06-04T09:22:37Z

Commit:

SHA: 52722b800bf654d8fc6696865ed19291807da3a5
Author: Sas Swart
coder/coder@52722b8

Failure evidence:

=== FAIL: coderd/x/nats Test_localSub_init/CrossSubjectListenerIsolation (25.06s)
    pubsub_internal_test.go:317:
        Error:       Condition never satisfied

The failing line is the require.Eventually waiting for fastCount >= total (line 317 in coderd/x/nats/pubsub_internal_test.go).

Notes:

No data race, panic, or OOM indicators observed in this job’s logs.
File history still points to 5cdc9e28a984 (“feat: add nats cluster peer support”) as the most recent substantive change to this test, so current assignment still looks correct.

fix: fix flakiness in nats localsub test (#26084)

Navigation Menu

Repository navigation

flake: Test_localSub_init/CrossSubjectListenerIsolation #1552

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

flake: Test_localSub_init/CrossSubjectListenerIsolation #1552

Description

Activity

flake-investigator commented on May 28, 2026

flake-investigator commented on May 28, 2026

flake-investigator commented on May 31, 2026

flake-investigator commented on Jun 1, 2026

flake-investigator commented on Jun 2, 2026

flake-investigator commented on Jun 2, 2026

flake-investigator commented on Jun 3, 2026

flake-investigator commented on Jun 4, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions