fix: regenerate invalidated modules in whole-repository-mode --update runs - #100
Conversation
… runs
first_module_tree.json is only ever written by the LLM clustering path,
so it stays {} for any repo small enough to skip clustering
("whole-repository documentation mode"). generate_module_documentation()
computed the processing order exclusively from that file, so once
--update/--compare-to invalidated (deleted) an affected sub-module's .md
via module_tree.json, the regeneration loop had nothing to iterate and
the run always ended in IncompleteGenerationError.
Fall back to module_tree for the processing order only when
first_module_tree is empty. The normal clustered-repo path is untouched,
since first_module_tree is non-empty there and the fallback never
triggers.
Fixes FSoft-AI4Code#99
54615ec to
e27accc
Compare
…ent-inserted sub-modules The fallback added for FSoft-AI4Code#99 only kicked in when first_module_tree.json produced an empty order, so it fixed whole-repository mode but left the same failure in clustered mode: first_module_tree.json never learns about the sub-modules agents insert into module_tree.json while documenting a complex module. Invalidating one of those with --update/--compare-to deleted its .md (plus the parent and overview), the loop only revisited the parent, and the run ended in IncompleteDocumentationError. Order from module_tree.json unconditionally instead. It is always a superset of the first tree, and modules whose .md already exists short-circuit in run_module_agent / generate_parent_module_docs, so plain re-runs still cost no LLM calls. This makes resolve_processing_order unnecessary, so it is removed. Tests: replace test_processing_order_fallback.py with test_processing_order_update.py, which adds fake-backend integration tests for the clustered nested case, the whole-repo case from FSoft-AI4Code#99, and a no-op re-run. get_processing_order / is_leaf_module become staticmethods since they use no instance state.
|
Thanks for the fix and the clear write-up in #99. I pushed a follow-up commit (fd060a3) to this branch that generalizes it. Why The fallback only fires when the order from What changed
Not addressed here Regenerated modules reuse the |
Fixes #99
Problem
--update --compare-to <commit>never regenerates an invalidated modulewhen the repo went through whole-repository documentation mode (module
clustering skipped because the repo fits in one context window). It
correctly deletes the affected module's
.md, then fails every time withIncompleteGenerationError. See #99 for the full trace of why.Root cause
generate_module_documentation()computed the processing order only fromfirst_module_tree.json, which stays{}forever for whole-repo-moderepos (only the LLM clustering path writes it) — even though
module_tree.jsoncan have real sub-modules a prior whole-repo agent runinserted. So the regeneration loop had nothing to iterate over for an
invalidated module.
Fix
Extracted the processing-order lookup into
DocumentationGenerator.resolve_processing_order(), which falls back tomodule_treeonly whenfirst_module_treeproduces an empty order. Thenormal clustered-repo path is unaffected, since
first_module_treeisnon-empty there and the fallback never triggers.
Testing
tests/test_processing_order_fallback.py(4 unit tests, no LLM/network needed) covering: fallback when
first_module_treeis empty,first_module_treestill wins when non-empty (regression safety forthe normal path), both-empty doesn't crash, and nested children are
still walked leaf-first through the fallback.
tests/test_overview_structure.pyagainst this change — all pass.
--update --compare-tonever regenerates invalidated modules in whole-repository documentation mode (always ends inIncompleteGenerationError) #99 (tinyauth/payment/notification TypeScript fixture, 2 commits): before the
fix, the run reproducibly failed with
IncompleteGenerationError;after the fix,
generate --update --compare-to <sha>succeeds, thechanged module regenerates, the two unaffected modules are correctly
skipped ("✓ Module docs already exists"), and the overview is rebuilt
referencing all three.