test_uci_data_acc.py exits 0 having run nothing on a non-UCI build (involuntary skip, the class #158 closed) #165

Symptom

tools/test_uci_data_acc.py exits 0 while running none of its assertions when the PRG in build/ is not a BACKEND=uci build:

SKIPPED [3] tools/test_uci_data_acc.py:720: uci_drain_resp is not in
build/labels.txt — this is not a BACKEND=uci build; rebuild with
`make clean && make BACKEND=uci USE_NISTCURVES_ONCHIP=1`
...
44 passed, 3 skipped in 5.52s

Exit code 0. A runner — and pytest is now a runner for this file, since it is pinned in pytest.ini testpaths — reads that as healthy.

Why this is the class #158 just closed

This is an involuntary skip: a missing label because the wrong build is present. The standard the project adopted in #158, taken from audit commit 7497e48, is:

an involuntary skip is a failure, an explicit skip is allowed but must never be silent

#158 fixed six instances of exactly this. This one arrived the same day, via #157, and was not in that sweep because it did not exist when the sweep was written.

The reference implementations are tools/test_x509.py (skipped groups added to total_failed) and tools/test_finished_verify.py:320-324 (missing label is FATAL, never a skip).

Mitigating detail, and why it is still not enough

The skip is loud and actionable — it names the missing symbol and prints the exact rebuild command, which is much better than the silent return 0s #158 removed. On a UCI build the three assertions run and pass.

But loudness is not the property that matters. The exit code is the only channel a runner reads, and pytest -q collapses the message. Someone running the suite against an ip65 build, or against a stale build/, gets green and no signal that the UCI accept-protocol assertions never executed — on a test whose whole purpose is to pin the DATA_ACC semantics fixed in #144.

Fix

Fail rather than skip when the label is absent. Either raise, or count it against the total the way test_x509.py:773 does. It is a small change to tools/test_uci_data_acc.py:720.

Consider also whether pytest.ini is the right home for a module that requires a particular backend's PRG. Everything else pinned there is pure-logic and build-independent; this one silently depends on what happens to be sitting in build/, which is precisely the coupling pytest.ini's own header comment says the pinned set exists to avoid.

Related

• test: six suites could exit 0 having tested nothing #158 — the six-instance sweep this belongs to
• uci: DATA_ACC is an accept, not a per-byte advance (#144 item 1) #157 — the PR that introduced it
• Three VICE suites pass or fail for reasons unrelated to what they measure (test_finished_verify's 16 negatives are vacuous on UCI) #161 — the broader "passes/fails for reasons unrelated to what it measures" cluster