feat: integrate ephemeral TCode prefix reuse into builds - #159
Merged
Merged
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
SkyeAv
added this pull request to stack #162
September 15, 2026 17:37
SkyeAv
changed the title
SkyeAv
force-pushed
the
runcache/integration
branch
from
September 15, 2026 17:51
d84d982 to
2f0c679
Compare
SkyeAv
marked this pull request as ready for review
September 15, 2026 18:06
SkyeAv
force-pushed
the
runcache/integration
branch
from
September 15, 2026 18:25
2f0c679 to
6508360
Compare
Wire content-addressed shared-prefix planning into the six-stage build while preserving the declarative TCode operation pattern and original output order: - compile_subgraph gains keyword-only resume/snapshot seams. Consumers seed a LazyFrame, skip the planned prefix, and emit one cache phase; producers invoke snapshots at guarded checkpoints. Defaults remain the historical full reduction. - Stage 5 filters Tcode.collect Path quick-exits before planning, creates RunCache only when RunPlan.shares, executes producer-first, fails loudly on a missing consumer snapshot, and reassembles subgraphs by original section index before compile_graph. The lazy import in lib.py avoids runcache's lib-keyed OP_COST import cycle. - Resume bounds (negative, empty, >= len(tcode)) raise coded runcache-invalid-resume; cache misses raise runcache-missing-snapshot. The TemporaryDirectory is cleaned on normal and exception paths. - Real-build tests prove shared source/resolve work executes once, stores match a no-sharing control, cache creation is skipped for trivial/no-sharing builds, output order is preserved, quick-exit mapping remains correct, and failure cleanup is complete. Verified: full uv run pytest -q -> 1379 passed, 43 skipped; focused + impact suites -> 337 passed; Tier-2 independent review PASS; ruff check/format clean; pyright 0 errors.
SkyeAv
force-pushed
the
runcache/integration
branch
from
September 15, 2026 18:36
6508360 to
afd2354
Compare
SkyeAv
added a commit
that referenced
this pull request
Sep 15, 2026
Cut 19.0.0 and bump the package version in pyproject.toml, CITATION.cff, and uv.lock. Major: five breaking changes ship. The `tablassert` console command now requires the optional `[cli]` extra, since `cyclopts` and `rich` left the base install (#187). Level-one normalization redefines fullmap keys as cleaned Unicode-lowercase, Porter2-normalized, byte-ordered token sets and moves the database to schema v6, so every schema-v5 fullmap is rejected and must be rebuilt (#171, #172, #174, #175, #176). `build-fullmap --aria2c` is gone -- aria2c is used automatically whenever the `[aria2]` extra is installed (#178) -- and `--taxon-allowlist` is gone because the built-in top-100 experimental-taxon allowlist now applies to every build, guarding both reuse paths by `META.taxon_allowlist` identity (#153). Logging now requires the `[log]` extra and is fully disabled without it (#152). Features: `tablassert.net` is a new stdlib-only transient/permanent classification and retry seam shared by the agent and the BABEL downloader (#163, #164), with one bounded jittered retry layer across the inner agent, reflexion, and judge (#167) and machine-readable `error_code` values on skipped checkpoint records (#166). `build-kg` gained an automatic ephemeral shared-prefix TCode cache keyed by content-addressed XXH64 op digests (#155-#159, #161). The `--distill` corpus became schema-uniform v2 with a sibling `outcomes.ndjson`, a deterministic `RewardConfig`-tunable reward, and the new zero-dependency `tablassert distill-weigh` command (#180-#183). `CLASS_FIELD_OVERRIDES` grants the DAKP sparse qualifier stack on the pinned association classes (#188). Fixes: `distill-export` partitions its corpus by content and unions the schema across record files, so a mixed v1/v2 directory no longer CastErrors or silently stringifies a column (#184); BABEL retry warnings survive the loguru sink; PMC downloads are idempotent, atomic, and bounded-parallel (#165); reflexion and judge prompts are hard-bounded (#168). Changelog: - versioned the Unreleased section as 19.0.0 - 2026-09-15 - merged three duplicate `### Added` blocks into one - moved the two `BREAKING:` entries from `### Changed` into `### Breaking Changes` and added a `**Migration:**` note to all five - added PR links to all 17 entries, which carried none - added the missing entries: the distill v2 corpus/reward/distill-weigh stack (#180-#183), the distill-export schema-drift fix (#184), the TCode run-cache detail (#155-#161), and the README badge removal (#154) Docs: none needed here. Every shipped doc change landed with its own PR, and the docs source-of-truth gate passes against the bumped tree. Testing: - make check -> exit 0 - uv run ruff check . -> All checks passed! - uv run pyright -> 0 errors, 0 warnings, 0 informations - uv run pytest -> 1622 passed, 52 skipped - cargo test --manifest-path rust/Cargo.toml -> 156 passed, 0 failed - cargo clippy --all-targets -- -D warnings -> clean - uv lock --check -> resolved 169 packages, lock current - uv run mkdocs build --strict -> exit 0 - docs SSOT + CLI coverage after the changelog edit -> 204 passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Layer 5 of 7 in the TCode run-cache stack (stack #162): wires digests, guards, planner, and
RunCacheinto the six-stage build. When sections share an expensive instruction prefix, the first section (producer) computes and snapshots it once and the rest resume from the snapshot — shared work executes exactly once per build, the cache directory is deleted when the build ends (including failure), and graph artifacts stay byte-identical.Integration
compile_subgraphgains keyword-onlyresume/snapshotseams (src/tablassert/lib.py); defaults reproduce today's behavior exactly, andagent.py's_reduce_opsmirror is untouched. Consumers seed the snapshot, skip the planned prefix, and show a singlecachephase tag.src/tablassert/cli.py) filtersTcode.collectPathquick-exits before planning (index-desync hazard), plans, opens oneRunCacheonly whenRunPlan.shares, runs producers first, reassemblessubgraphsin original section order beforecompile_graph, and deletes the cache on exit.runcache-invalid-resume; a planned consumer whose snapshot is missing raisesruncache-missing-snapshot(no silent recompute).lib.pyimportsruncachelazily —runcachekeysOP_COSTonlibcallables, so a top-level import would cycle.--headconsumers reuse the producer's random sample;--loglines from a shared label-free prefix attribute to the producer section.Testing
tests/test_run_cache_e2e.py, tiny real redb): two sections sharing a source+resolve prefix execute it once (op-call counting), stores are byte-identical to a no-sharing control build, zero-sharing builds create no temp directory, output order is preserved, quick-exit mapping is intact, and the cache dir is gone on failure.uv run pytest -q(full suite) → 1379 passed, 43 skipped--no-cov -n 0) → 337 passeduv run ruff check .→ passed;uv run ruff format --check .→ formatted;uv run pyright→ 0 errors; independent reviewer pass