flake: TestRun (natsbench multiple variants) #1612

CI Failure Details

CI Run: https://github.com/coder/coder/actions/runs/28496003256
Failed Job: https://github.com/coder/coder/actions/runs/28496003256/job/84462298128 (test-go-pg (windows-2022))
Failure Time: 2026-07-01T05:52:31Z

Commit Info

• SHA: cb1a87b9c0d0d04e4e1079a2f0c1203904c4f27b
• Author: Jake Howell
• Commit: coder/coder@cb1a87b

Failing Tests

• TestRunSingleNode
• TestRunCluster

File: coderd/x/nats/natsbench/natsbench_internal_test.go (around lines ~43-105)

Error Output

=== FAIL: coderd/x/nats/natsbench TestRunSingleNode (0.03s)
    natsbench_internal_test.go:69:
        Error:      "0" is not greater than "0"

=== FAIL: coderd/x/nats/natsbench TestRunCluster (1.26s)
    natsbench_internal_test.go:104:
        Error:      "0" is not greater than "0"

Error Analysis

Both tests assert non-zero throughput (res.PubsPerSec / res.DeliveriesPerSec) after running the benchmark. The results reported 0, causing require.Greater(..., 0.0) to fail for both single-node and cluster runs.

Root Cause Assessment

Likely flaky timing/benchmark instability on Windows runners where the benchmark reports zero throughput despite successful setup. No evidence of infra-level failure beyond the zero metrics.

Data Race / Panic / OOM Checks

Searched the job logs for WARNING: DATA RACE, panic:, out of memory, signal: killed — none found.

Assignment Analysis

Intended blame targets:

• git blame -L 43,72 coderd/x/nats/natsbench/natsbench_internal_test.go
• git blame -L 74,104 coderd/x/nats/natsbench/natsbench_internal_test.go

File history:

• git log --oneline -10 --follow coderd/x/nats/natsbench/natsbench_internal_test.go
  • Most recent substantive change: ea1379d42 ("chore: add nats benchmarking pkg") by Jon Ayers (PR #26396, authored by @sreya).

Assigning to @sreya as the current owner of the natsbench test area (original PR author and related NATS test owner).

Related Issues Search

Searched coder/internal for:

• TestRunSingleNode
• TestRunCluster
• natsbench_internal_test.go
• natsbench

No matching issues found.

Reproduction

go test ./coderd/x/nats/natsbench -run TestRun -count=1