Complete the derived YAML tree-sitter target (grammar doesn't generate) #3

Monogram derives four targets from each grammar definition: a TextMate grammar, a Monarch grammar, a tree-sitter grammar, and the CST parser. For yaml.ts, three of the four work — the TextMate highlighter (93.8% vs the parser oracle, ahead of RedCMD's 92.4%), the Monarch grammar, and the CST parser (src-coverage 73.4% vs the yaml package). The tree-sitter target is unfinished: tree-sitter/yaml/grammar.js does not generate.

This is not a regression and nothing depends on it today — CI builds and gates only the TypeScript and HTML tree-sitter grammars (the tree-sitter thesis is carried by TS at 95.9%, ahead of the official grammar), and the "generate every derived grammar" conflict gate in ci.yml explicitly excludes yaml. Filing this to track the work for if/when YAML-as-a-tree-sitter-target becomes a goal.

Three things block it

1. Structural indent tokens are emitted as token(//) (invalid JS). INDENT / DEDENT / NEWLINE and the block-scalar body are engine-emitted placeholder tokens — their grammar pattern is the never-match (?!). sanitizeTreeSitterRegex strips the zero-width assertion to empty, so they serialize as token(//): an empty regex literal, which JS parses as a line comment, so grammar.js won't even load.
   Fix: route them to tree-sitter externals (the way the HTML markup path already handles raw_text), keyed off grammar.indent in planScannerTokens (src/gen-treesitter.ts). Prototyped — it makes grammar.js load and keeps every other language byte-identical — but it is necessary, not sufficient (see Fix template interpolation opener scanning #2).

2. The node rule matches the empty string. A YAML node can be null (key: with no value), so Node is nullable, and tree-sitter generate rejects empty-matching non-start rules: "the rule node matches the empty string." Resolving it needs nullable-rule elimination — make node non-empty and wrap its references in optional(...) — either by restructuring yaml.ts (must not regress the parser's 73.4% or the TextMate highlighter) or as a general transform in gen-treesitter.ts. Further conflicts may surface after this.

3. The C external scanner is a stub. tree-sitter/yaml/src/scanner.c does not implement YAML indentation — scan() returns false. A real scanner needs an indent stack with serialize / deserialize, flow-collection suspension, and block-scalar bodies, mirroring the indentation logic in src/gen-lexer.ts.

Why it would be worth doing

It extends the tree-sitter thesis proof to an indentation-sensitive grammar: the same non-deterministic grammar a flat-LR engine chokes on, tree-sitter's GLR absorbs — even for YAML's indentation. It would also let yaml join the CI conflict gate and remove the committed invalid-JS grammar.js.

Acceptance

• cd tree-sitter/yaml && npx tree-sitter generate && npx tree-sitter build --wasm . succeeds.
• yaml is added to the "generate every derived grammar" step in ci.yml.
• Ideally a yaml tree-sitter accuracy bench (there is none today; the only validation would be parse correctness over a corpus).