Fix/reasoning token cost accounting

[yl231]

Summary

Fixes two cost-accounting gaps in the router evaluator, both reported in #135 by @namitha-sqwish. Both are confirmed and reproduce exactly against the committed prediction files.

These are distinct from the empty-response fix in #118 — see Issue 2 below for how they slip through that gate.

Issue 1 — Reasoning tokens were never billed

ModelEvaluator.calculate_inference_cost charged only input_tokens + output_tokens. Reasoning/"thinking" tokens — which providers (xAI, OpenAI, …) bill at the completion rate but report only implicitly as total_tokens − (input + output) — went entirely uncounted.

Fix: bill the recovered reasoning tokens at the model's output rate (or a reasoning_token_price_per_million field when configured).

Uncounted reasoning tokens across leaderboard rows:

Router	Affected rows	Uncounted tokens
Azure-Model-Router	4,803	8,603,765
Nadir	3,101	6,449,941
vLLM-SR	802	2,007,419

Issue 2 — Successful generations with no usable token usage were free

Rows with success: true and a real, non-empty answer but token_usage: {} or output_tokens: 0 passed the success-only validity gate from #118 and were therefore scored as correct at $0 cost.

This is the inverse of the #118 exploit and slips through it by construction: #118 catches empty/failed responses, but these rows are successful with genuine answers — they only lack reportable usage. success is True waves them through, the scorer counts them correct, and empty token_usage makes calculate_inference_cost return 0.0.

Fix: the validity gate now also requires output_tokens > 0. Such rows are treated as failed inference (scored as wrong, excluded from cost) consistently across the RouterArena score, cost aggregation, and the optimality pool. This extends #118 rather than duplicating it.

Affected (forwarded to the router authors):

• vLLM-SR: 279 rows (278 grok-4-1-fast-reasoning with empty token_usage, 1 gemini-3.1-flash-lite with output_tokens: 0)
• Sqwish: 1 row (deepseek-v4-flash, output_tokens: 0)

Changes

• llm_evaluation/evaluate_models.py — bill reasoning tokens at the output rate.
• llm_evaluation/run.py — has_usable_token_usage / is_valid_generation helpers; failed-inference rows excluded from accuracy credit and cost across compute_router_metrics, _build_evaluation_dict, and compute_optimality_from_predictions.
• router_inference/compare_router_accuracy.py — mirror the reasoning-token fix.
• router_inference/check_config_prediction_files.py — submission-time validation: a successful non-empty generation must report output_tokens > 0.
• tools/audit_token_accounting.py — audit script reporting per-router reasoning-token magnitude and failed-inference counts.
• scripts/rescore_affected.sh — reproducible re-score driver.
• README.md — leaderboard re-scored (below).

Leaderboard impact

Only routers whose models emit reasoning tokens or zero-usage rows are re-scored (at current model_cost.json prices); all others are unchanged.

Router	Arena	Accuracy	Cost/1K
vLLM-SR	75.38 → 72.15	75.97 → 73.19	$0.11 → $0.23
Nadir	73.33 → 72.29	74.87 → 75.01	$0.29 → $0.68
Azure	71.87 → 70.42	72.82 → 72.94	$0.22 → $0.73

New top of board: 🥇 Sqwish · 🥈 AgentForge · 🥉 Weave · 4 Nadir · 5 vLLM-SR.

vLLM-SR's accuracy drop is driven by Issue 2: 238 of its 279 zero-usage rows had previously been scored correct and are now counted as wrong (the denominator is unchanged at 8,400).

Notes for reviewers

Resolves #135.