Skip to content

feat!: use aria2c automatically when the [aria2] extra is installed - #178

Merged
SkyeAv merged 4 commits into
mainfrom
aria2-default
Sep 15, 2026
Merged

SkyeAv merged 4 commits into
mainfrom
aria2-default

Conversation

@SkyeAv

@SkyeAv SkyeAv commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Makes build-fullmap pick its downloader automatically — the bundled aria2c binary whenever the optional [aria2] extra is installed, Tablassert's Python downloader otherwise — and removes the --aria2c / -a flag entirely. Users who install the extra now get the faster download with no flag to discover or remember.

Automatic downloader selection

  • Single decision point: build_fullmap resolves extras.is_installed("aria2") exactly once and threads the result into fetch_prebuilt_fullmap and build_fullmap_pipeline through their existing aria2c keyword parameter — no per-file probing, no duplicated decision logic.
  • Probed after the reuse short-circuit: a run that reuses an existing database downloads nothing, so it neither probes the extra nor says anything about downloaders.
  • Announced, not just logged: the choice is emitted as one download_logger.info record and one stderr line (tablassert build-fullmap: the [aria2] extra is installed; using the bundled aria2c for downloads). loguru's console sink only exists inside a pipeline stage and file logging needs the [log] extra, so the log record alone would never reach a normal terminal; the stderr line follows the reuse short-circuit's existing print(..., file=sys.stderr) precedent. Both derive from one string so they cannot drift.

Flag removal (BREAKING)

  • --aria2c / -a are gone from build-fullmap; passing either is an unknown-option parse error, matching the --taxon-allowlist removal precedent in the current Unreleased entry.
  • The exit-2 preflight is gone with it — a missing [aria2] extra is no longer an error condition, just a downloader choice.
  • No replacement knob: there is deliberately no flag, env var, or config key to opt out of aria2c when the extra is installed. Uninstall the extra instead.

Design

  • Detection via extras.is_installed("aria2") (the existing find_spec probe) rather than a _resolve_aria2_binary() try/except: it is cheap, does not execute the module, and lets the choice be logged once at the command boundary. download_babel_file_aria2c's late import_module remains the loud safety net for a present-but-broken extra (TOCTOU).
  • Failure semantics are unchanged. A failed prebuilt download still surfaces as PrebuiltFullmapUnavailable → warning → from-scratch BABEL build (and that scratch build keeps using aria2c); a failed BABEL-loop download still propagates BabelDownloadError loudly. There is deliberately no new mid-download fallback from a failing aria2c to the Python downloader — that would mask real network or binary failures.
  • Accepted caveat — no platform gate. Selection is a bare find_spec and the aria2 extra carries no environment marker, so a source-built aria2 on macOS would be used. The docs therefore say a normal macOS install resolves to the Python downloader rather than claiming macOS "always" does.
  • Honest resolution-failure messages. aria2_unavailable_detail() is reachable from the CLI only after the is_installed probe said yes, so "install the [aria2] extra" was a dead end there (pip reports already-satisfied). It now splits on install state: a present-but-broken install is named as such with reinstall/uninstall advice, while library callers with a genuinely absent extra keep the install hint. The wording is "cannot be resolved (missing, or lacking ARIA2C)" because _resolve_aria2_binary also converts a shadowed module's AttributeError into ImportError.
  • Deferred: the adjacent allowlist-rebuild warning has the same terminal-invisibility problem this PR fixes for the downloader choice; errors.py still appends "Check network connectivity or pin a different BABEL version" to broken-install errors. Both are pre-existing and out of scope.

Docs

  • docs/cli.md — drops the --aria2c option row; the example is now install-the-extra plus a plain build-fullmap, with a paragraph on automatic selection.
  • docs/fullmap.md — opt-in prose replaced throughout; clarifies that both download stages (the prebuilt fullmap.tar.zst archive and the from-scratch BABEL files) use aria2c.
  • docs/installation.md — extras row and macOS note reworded; the preflight table intro flags that the [aria2] row is a downloader probe, not a failure report. [aria2] stays named in "When an extra is missing" because test_installation_preflight_docs_cover_source_preflight_calls AST-walks the surviving is_installed call site.
  • README.md extras row reworded; CHANGELOG.md carries a BREAKING Unreleased entry. Historical changelog entries mentioning --aria2c are intentionally left as history.

Tests

  • Auto-selection, both directions: the choice test is parametrized over both extras.missing states, pinning the exact single log line, its ordering, and that the rejected downloader never runs.
  • Failure semantics pinned: a prebuilt aria2c failure still falls back to a scratch build with aria2c still selected, and a BABEL-loop aria2c failure still propagates — both with the Python downloader patched to pytest.fail, so a silent rescue fails the test.
  • No-op silence: an existing database neither probes the extra nor emits a choice line on stderr.
  • Stale-docs guard: a new parametrized test fails if any live surface (docs/**/*.md, README.md, llms.txt, CONTRIBUTING.md) mentions a retired flag; CHANGELOG.md is excluded since history may name them. --aria2c is matched globally, while the -a table-cell shapes are scoped to the owning command's section — line-scoping would have been vacuous, since option-table rows never repeat the command name — with self-checks proving the matcher is non-vacuous.

Testing

  • uv run pytest1404 passed, 43 skipped (baseline on main: 1361 passed)
  • uv run ruff check .All checks passed!; uv run ruff format --check .88 files already formatted
  • uv run pyright0 errors, 0 warnings, 0 informations
  • uv run tablassert build-fullmap --help 2>&1 | grep -c -- '--aria2c'0
  • uv run tablassert build-fullmap --aria2c → exit 1, Unknown option: "--aria2c".; same for -a
  • rg -n -- '--aria2c' docs/ README.md llms.txt CONTRIBUTING.md src/ → zero matches
  • rust/ untouched — every aria2 call site is Python-side

…ra is installed

Remove the --aria2c/-a flag from build-fullmap: the downloader is now chosen
automatically — the bundled aria2c binary when the [aria2] extra is installed,
the Python downloader otherwise. The choice is resolved once after the reuse
short-circuit (no-op runs stay silent), logged at info level either way, and
threaded to fetch_prebuilt_fullmap and build_fullmap_pipeline. Failure
semantics are unchanged: prebuilt download failures still fall back to a
from-scratch build and BABEL-loop failures still propagate loudly; aria2c is
never silently rescued by the Python downloader.

Verified: uv run pytest tests/test_cover_cli.py -k 'aria2 or fullmap' -n 0 --no-cov -q
(42 passed); full test_cover_cli.py (55 passed); --help lists no --aria2c;
--aria2c and -a exit 1 as unknown options; ruff check + format clean.
…ed flags

Update every live surface for the --aria2c/-a removal: docs/cli.md drops the
flag row and shows install-extra + plain build-fullmap; docs/fullmap.md and
docs/installation.md describe automatic selection (macOS always uses the
Python downloader — no wheels); README.md extras row reworded; CHANGELOG
Unreleased carries the BREAKING entry naming flag removal, automatic use, the
removed exit-2 preflight, and unchanged failure semantics. Adds a stale
-reference guard: a new parametrized test fails if any of docs/**/*.md,
README.md, llms.txt, or CONTRIBUTING.md mentions a retired CLI flag
(CHANGELOG.md excluded — history may name retired flags).

Verified: uv run pytest tests/test_docs_source_of_truth.py -n 0 --no-cov -q
(154 passed, includes 19 new guard params); rg -- '--aria2c' across live
surfaces -> zero matches; uv run pytest tests/test_cover_cli.py -k aria2
(17 passed); ruff check + format clean.
…oader choice

Remediation of the independent audit findings:

- aria2_unavailable_detail()'s macOS branch claimed the Python downloader
  "takes over"; that branch only fires from download_babel_file_aria2c's
  ImportError path, which raises BabelDownloadError loudly and is never
  rescued (the scratch fallback re-enters the same helper with aria2c=True).
  The message now names the broken install and the real fix: uninstall it.
- The resolved downloader choice was logged only, so it never reached a
  normal terminal (loguru's console sink exists only inside a pipeline
  stage; file logging needs [log]). build-fullmap now also announces the
  choice on stderr, following the reuse short-circuit's precedent, and
  stays silent on a no-op run.
- Parametrized the choice-line test over both extras.missing states so the
  aria2c-selected info line is pinned exactly-once too (filter matches
  "[aria2] extra" — "downloader" never matches the positive wording).
- RETIRED_CLI_FLAGS also guards the `-a` option-table cell shapes.
- Docstring/doc nits: fetch_prebuilt_fullmap says bundled binary; stale
  opt-in prose removed from tests; installation.md notes the aria2 row is
  a probe, not a failure report; fullmap.md names both download stages.

Verified: uv run pytest -> 1400 passed, 43 skipped (baseline 1361 + 39:
+1 parametrized choice test, +38 guard params); ruff check + format clean;
pyright exit 0; build-fullmap --help has no --aria2c; --aria2c exits 1;
rg -- --aria2c across live surfaces -> zero matches.
… flag guard

Second-round audit remediation:

- aria2_unavailable_detail() claimed the aria2c module was "unimportable";
  _resolve_aria2_binary also raises for a module that imports but exposes no
  ARIA2C (a shadowed aria2c), so the wording is now "cannot be resolved
  (missing, or lacking ARIA2C)". The docstring also qualifies the
  "fails the same way" claim: a warm download cache returns before the
  resolver and --force skips the prebuilt attempt, so those paths never
  reach it — the core no-rescue claim is unchanged.
- The non-macOS branch was premise-stale: the CLI reaches it only after its
  own is_installed probe said yes, so "install the [aria2] extra" was a dead
  end (pip reports already-satisfied). It now names the broken install and
  suggests reinstall/uninstall, while library callers with a genuinely
  absent extra keep the install hint. Both branches are pinned by a
  parametrized test that bans the other state's wording.
- docs no longer claim macOS "always" uses the Python downloader — the
  selection is a bare find_spec with no platform gate, so the wording is now
  "a normal macOS install resolves to" it.
- fullmap.md clarifies that only the from-scratch pipeline's BABEL stage is
  meant (the prebuilt archive uses aria2c too).
- build_fullmap derives both the log record and the stderr announcement from
  one string so they cannot drift.
- Corrected a false rationale in the choice-line test's docstring, and made
  the retired-flag guard command-section-scoped for short flags (--aria2c
  stays global) with non-vacuity self-checks, so a future legitimate -a
  short flag on another command cannot trip it.

Verified: uv run pytest -> 1404 passed, 43 skipped; ruff check + format
clean; pyright exit 0; build-fullmap --help has no --aria2c; --aria2c exits
1; rg -- --aria2c across live surfaces -> zero matches.
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 10fb7564-4bb0-4357-bfb5-c649e8fd688d


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@SkyeAv
SkyeAv merged commit 4092030 into main Sep 15, 2026
5 checks passed
@SkyeAv
SkyeAv deleted the aria2-default branch September 15, 2026 17:42
SkyeAv added a commit that referenced this pull request Sep 15, 2026
Cut 19.0.0 and bump the package version in pyproject.toml, CITATION.cff,
and uv.lock.

Major: five breaking changes ship. The `tablassert` console command now
requires the optional `[cli]` extra, since `cyclopts` and `rich` left the
base install (#187). Level-one normalization redefines fullmap keys as
cleaned Unicode-lowercase, Porter2-normalized, byte-ordered token sets and
moves the database to schema v6, so every schema-v5 fullmap is rejected and
must be rebuilt (#171, #172, #174, #175, #176). `build-fullmap --aria2c`
is gone -- aria2c is used automatically whenever the `[aria2]` extra is
installed (#178) -- and `--taxon-allowlist` is gone because the built-in
top-100 experimental-taxon allowlist now applies to every build, guarding
both reuse paths by `META.taxon_allowlist` identity (#153). Logging now
requires the `[log]` extra and is fully disabled without it (#152).

Features: `tablassert.net` is a new stdlib-only transient/permanent
classification and retry seam shared by the agent and the BABEL downloader
(#163, #164), with one bounded jittered retry layer across the inner agent,
reflexion, and judge (#167) and machine-readable `error_code` values on
skipped checkpoint records (#166). `build-kg` gained an automatic ephemeral
shared-prefix TCode cache keyed by content-addressed XXH64 op digests
(#155-#159, #161). The `--distill` corpus became schema-uniform v2 with a
sibling `outcomes.ndjson`, a deterministic `RewardConfig`-tunable reward,
and the new zero-dependency `tablassert distill-weigh` command (#180-#183).
`CLASS_FIELD_OVERRIDES` grants the DAKP sparse qualifier stack on the pinned
association classes (#188).

Fixes: `distill-export` partitions its corpus by content and unions the
schema across record files, so a mixed v1/v2 directory no longer CastErrors
or silently stringifies a column (#184); BABEL retry warnings survive the
loguru sink; PMC downloads are idempotent, atomic, and bounded-parallel
(#165); reflexion and judge prompts are hard-bounded (#168).

Changelog:
- versioned the Unreleased section as 19.0.0 - 2026-09-15
- merged three duplicate `### Added` blocks into one
- moved the two `BREAKING:` entries from `### Changed` into
  `### Breaking Changes` and added a `**Migration:**` note to all five
- added PR links to all 17 entries, which carried none
- added the missing entries: the distill v2 corpus/reward/distill-weigh
  stack (#180-#183), the distill-export schema-drift fix (#184), the
  TCode run-cache detail (#155-#161), and the README badge removal (#154)

Docs: none needed here. Every shipped doc change landed with its own PR,
and the docs source-of-truth gate passes against the bumped tree.

Testing:
- make check -> exit 0
- uv run ruff check . -> All checks passed!
- uv run pyright -> 0 errors, 0 warnings, 0 informations
- uv run pytest -> 1622 passed, 52 skipped
- cargo test --manifest-path rust/Cargo.toml -> 156 passed, 0 failed
- cargo clippy --all-targets -- -D warnings -> clean
- uv lock --check -> resolved 169 packages, lock current
- uv run mkdocs build --strict -> exit 0
- docs SSOT + CLI coverage after the changelog edit -> 204 passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants