fix(analyzer): keep Python and Perl syntax from exhausting the shell parser

[rng1995]

Summary

Valid Perl and Python source makes has_bounded_parse_exhaustion (static_patterns_tool_misuse) report static_parse_limit. The file is then recorded as partially inspected, and every SKILL.md reference to it becomes a HIGH AE1, which blocks downstream signing gates even though the content is harmless. All triggers below are new in 2.12.0 (2.11.2 scans the same files complete).

This PR recognizes Perl eval BLOCK, and gives Python string, f-string and comment tokens ownership of their bytes when the whole module is proven to be valid Python. Refs #694 (Python part; the Rust, JavaScript, PowerShell and Markdown-table cases there are not addressed).

Customer impact

Reported internally against SkillSpector 2.12.0 (via SkillEvaluator 1.6.0). Two internal skills with larger Python/Perl helper scripts fail their release gate with a critical "scan did not execute reliably" result.

Skill	Files reported static_parse_limit	AE1
Skill A	4 helpers: 1 Perl, 3 Python. This PR clears all 4 (one more helper is fixed by #777).	7 → 1
Skill B	2 Python files	1 → 0

Neither skill needs to change.

Root cause

Source	Why it tripped
Perl eval { require $f; 1 } or die "x: $@";	_command_string_from_clause treats a clause-initial eval as shell eval. The dynamic $f operand makes the reparsed string unresolvable (None), which returns exhaustion immediately.
Python md = "\n```\n" followed by more than 4 KB of code	A backtick inside double quotes opens shell command substitution, which never closes. unresolved_end - start > _SHELL_COMMAND_WORD_CHARS with unresolved_end at the end of the view.
Python docstring/comment apostrophes and backticks (helper's get_token() , # ... the caller's path)	Same: the apostrophe opens a shell single quote that runs to EOF.
signal.alarm(timeout), if (timeout > 0 ...	A clause starting with the wrapper name timeout is reparsed as a command string.
# Invoke with $0 = the launcher's path	A runtime-selected command in a comment takes "operands" from the following code.

Fix

Perl. With file_type == "perl", a clause-initial bare eval followed by { is not a string reparse. { already starts a new clause, so the statements in the block are still scanned.

• Still partial: eval $code, eval "...", eval qq{...}, quoted "eval {" inside a shell string, and any shell-file eval.
• Unchanged: eval { `$TOOL -rf /` }, eval { qx(sh -c "$cmd") } and brace-expansion inside the block; eval { system("rm -rf /") } still produces TM1.

Python. A lazy _PythonSourceOwnership is used only for a complete .py view that:

• ast.parse accepts (tokenize alone accepts $ and backticks);
• is within MAX_PYTHON_AST_SOURCE_CHARS;
• has no lone CR;
• has no non-Python shebang.

Its string, f-string/t-string (outermost) and comment spans come from tokenize and are verified against the source. Valid Python code outside those tokens contains no shell quote or expansion opener, so:

• an unresolved shell word is charged only up to the first code-level word break after it, instead of the end of the view;
• a command-string clause (eval, sh -c, wrappers) that begins in Python code is not reparsed as shell;
• a runtime-selected command inside a comment is rechecked against that comment alone, because no shell receives a comment. # $TOOL -rf / stays partial.

Unchanged:

• Fragments, malformed source, shell-shebang polyglots, and any single string or comment that itself exceeds the 4096-character budget keep today's results.
• So do parsed.limited, the printf, destructive-rm, brace-expansion and root-glob checks, runtime commands built from strings, all shell and Markdown behavior, and every rule finding from analyze().

The new section in docs/ANALYSIS_RESOURCE_BOUNDS.md records the ownership contract.

Tests

tests/nodes/analyzers/test_host_language_shell_ownership.py has 56 tests; 32 of them fail on main, and all fail-closed controls pass on both.

• Perl: benign eval blocks (complete and fragment context) → False; string-eval forms → True; a shell-file control → True; commands inside the block still scanned and TM1 kept.
• Python benign → False, each paired with a shell-text control that is True: fence in a string or f-string, unpaired backtick in a docstring, docstring apostrophe with double backticks, comment apostrophe, the static_parse_limit on valid Rust, Python and POSIX shell source leaves files partially inspected #694 regex character-class case, end="", implicit concatenation, timeoutwrapper names, comment$0`. Also CRLF, no trailing newline, non-ASCII, and exact span offsets.
• Still → True: invalid Python, #!/bin/sh, lone CR, NUL, complete_context=False, a single literal/adjacent literal/comment over 4096 chars, a comment with $TOOL -rf /, and a .sh file with an unmatched backtick plus a long tail (ledger PARTIAL).
• Runtime deadline honored by the token scan.
• Scan level (graph.invoke): a SKILL.md referencing a Python and a Perl helper is complete with no AE1; the over-budget negative control stays partial with AE1.
• make lint and make format-check pass. The unit suite (pytest -m "not integration and not provider" tests/) gives 10343 passed, 15 skipped, 4 xfailed, 0 failed.
• End-to-end on the two affected skills: compared with main, the only changes are the removed static_parse_limit rows and their AE1s; no rule finding changes.

Residual

• eval { `sh -c "$cmd"` } was partial on main only incidentally, through the eval-operand path. It is now complete, the same as my $x = `sh -c "$cmd"` already is. The underlying gap is that command strings inside backtick substitutions are not recognized as clauses; it predates this PR.
• Python files over 256K characters, marker windows, and derived views that are not valid Python keep today's behavior.
• A Python module that starts with a UTF-8 BOM gets no token ownership. The file cache decodes with utf-8, not utf-8-sig, so the leading U+FEFF fails ast.parse, and main already reports such a file as syntax_error in the AST analyzers. A test pins today's conservative result; stripping the BOM belongs with a separate decoding fix.

🤖 Generated with Claude Code

[rng1995]

Review: the ownership model is sound. Requiring a full ast.parse, verifying spans against the source, and bounding at the first code-level break rather than at the token end are the right conservative choices. The Perl eval BLOCK gate on the exact bare word keeps quoted "eval {" in shell strings failing closed. I checked tab-indented, coding-cookie, Python 2 (falls back) and invalid-escape sources. Two P3 items, no blockers.