Registry: cancel race leaves a phantom queued job holding a capacity slot; a malformed registry.json stops the backend booting #520

Two verified defects in app/core/registry.py / app/pipeline/jobqueue.py.

1. Cancel racing the queue worker leaves a phantom "queued" job that permanently eats a capacity slot

The worker pops an id at jobqueue.py:213 and only claims it with _set_running(job_id) at :229. Between those, the job is in neither _queue nor _running_id. cancel_job runs in Starlette's threadpool, so it interleaves.

Interleaving:

1. Worker calls _pop_next() -- job is no longer in _queue.
2. cancel_job (app/api/jobs.py:331) sets job.cancel_requested = True.
3. jobqueue.discard(job_id) returns False (already popped), so cancel_job never sets status="cancelled", never calls cleanup_job_dir, never persists.
4. jobqueue.running_id() is still None, so the terminate branch is skipped too.
5. The worker reaches if job.cancel_requested ... continue (jobqueue.py:222-224) and "drops it silently".

The job stays in _jobs with status="queued" forever.

Verified consequences:

• Permanent capacity loss. pending_count (registry.py) is sum(1 for j in _jobs.values() if j.status == "queued" ...), so register_if_capacity counts it forever.
• Invisible. Not in _queue, not running, so it does not appear in the queue view.
• Disk not freed. An uploaded source (up to 400 MB) is never deleted.
• Resurrected on every restart. _RESUMABLE = {"queued", ...} and _PERSISTED = _TERMINAL | _RESUMABLE, so persist() writes it and restore() re-queues it.

The comment at jobqueue.py:226-228 reasons carefully about the pop-to-status-write window, but only for capacity accounting -- not for this path.

Fix: make the pop-and-claim atomic, or have cancel_job handle "not in queue and not running" by finalising the job itself.

2. A registry.json that is valid JSON but not an object prevents the backend from starting

registry.py:147: data = _migrate(json.loads(path.read_text(encoding="utf-8")))

_migrate calls data.get(...). For a top-level [], null, or a bare string that is an AttributeError, which is not in the except (OSError, json.JSONDecodeError, TypeError, ValueError) tuple, so it propagates.

restore_registry(JOBS_DIR) runs at module scope in app/main.py, so the import fails and the backend never starts, with no self-healing path and no way for a user to recover short of deleting the file by hand.

Verified additionally: jobs_dir.iterdir() and the trailing persist(jobs_dir) both sit outside the try block, so an OSError from either is fatal at startup too.

Fix: catch broadly around the whole restore, treat an unusable registry the way a missing one is treated (rename it aside and start clean), and bring the iterdir/persist calls inside the guard.

Test

• A cancel arriving in the pop-to-claim window finalises the job and frees its slot.
• restore() with registry.json containing [], null, "x", and a directory-permission error each start cleanly rather than raising.