Merge a called analyzer's lazy entries into known words (4.1.2) - #741
Merged
Merged
Conversation
A word already in memory with attributes was never looked up in lazily loaded *full dictionaries. For an analyzer's own lazy files that is intended: its in-memory dictionary entry wins. But a called analyzer shares its caller's KB, and the caller's dictionaries define many of the same words, so the called analyzer's en-full.dict never reached them: "hello" kept the caller's greeting=1 and got no part of speech. Lazy files that callanalyzer registers are now opened to merge (FullFile::merge). CG::mergeFullWord searches each of them once per word, whether or not the word is in memory or has attributes, and adds what they hold; the ordinary lazy search skips them. An analyzer's own lazy files behave as before. The fixture moves "red" to the callee's lazy shades-full.dict and gives the caller its own "red mood=angry" entry; red must end up with both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Follow-up to #740, reported by a session calling english-phrases-analyzer from client-reply-analyzer.
A word the calling analyzer's dictionary already defined got none of the called analyzer's lazy
*fullattributes. 4.1.1 made a word in memory with no attributes get one lazy lookup, but a word with attributes was still treated as done. The caller'sreplywords.dictdefineshello greeting=1,team salutation=1,on attribution=1and so on, so the called analyzer'sen-full.dictnever reached those words:hellokeptgreeting 1and got no part of speech (on its own, english-phrases-analyzer gives itint,noun,pos num 2).For an analyzer's own lazy files, skipping a word its dictionary defines is intended, so this is not changed globally. Instead:
callanalyzerare opened to merge:openFullDict/openFullKBBtakemerge(defaultfalse), stored onFullFilewith a per-file set of words already searched.CG::mergeFullWordsearches each merge file once per word, whether or not the word is in memory or has attributes, and adds what it holds (dict hits into the word itself; kbb-only hits linked asmeaning).findFullDictWord/findFullKBBWordskip merge files, so nothing is searched twice.An analyzer's own lazy files behave exactly as before. Version bumped to 4.1.2.
One side effect: when both dictionaries give a word the same value (e.g.
pos=noun), it is now stored twice, andpos numcounts it twice.Tests
The fixture moves
redto the callee's lazyshades-full.dictand gives the caller its ownred mood=angryentry; a new output line requiresred:angry yes. The 4.1.1 binary fails it (red is not colored and has nocolor); this build passes.Locally on Windows: all 15 fixture steps in
build-windows.ymlpass, including the rule-level debugger, and parse-en-us'sfinal.treeis byte-identical to the golden copy.Help: separate PR in VisualText/visualtext-files; merge it after this ships.
🤖 Generated with Claude Code