fix(bash-ast): stop deparse from changing shell semantics

[pyramation]

Summary

deparse(parse(src)) silently changed what a script does. Measured over the 111+ inline run: blocks of constructive-db/.github/workflows (constructive-io/constructive-planning#1646): 40 changed output, 3 failed to parse, 1 hung the parser. After this change all 114 extracted blocks round trip with an unchanged AST, an idempotent second deparse, and output accepted by bash -n.

Five root causes, all in the lexer/parser/deparser boundary:

• Here-doc bodies were lexed as shell. cat <<EOF\n|\nEOF spun forever. <</<<- now register a PendingHereDoc; the newline handler reads raw lines until the delimiter and hangs the body off the operator token, so the body is opaque text carried on Redirect.heredoc and re-emitted after the line that opened it.
• & was consumed as a plain separator, so kubectl proxy & became blocking. async now lives on the command node (BaseNode.async) and the deparser appends &.
• Brace groups were flattened to their inner CompoundList, so [ -n "$x" ] || { echo …; exit 0; } made exit 0 unconditional and { a; b; } | tee piped only b. New BraceGroup node, kept grouped through pipelines and &&/||; brace groups, subshells and compound commands can now carry redirects.
• Every NAME=value word was an ASSIGNMENT_WORD, which turned psql --set ON_ERROR_STOP=1 into a second statement and broke local label="$1". The lexer now tracks command position (;, &, |, &&, ||, newline, (, ), {, reserved words) and only classifies an assignment there or as an operand of export/local/declare/readonly/typeset.
• <(…) and {…} were split on punctuation. Process substitutions are read whole; {/} are reserved words only when standalone, so -o jsonpath={.metadata.labels} survives.

Two new options, both opt-in so existing behavior is unchanged:

parse(src, { keepComments: true, timeoutMs: 5000 })

keepComments emits COMMENT tokens and Comment nodes (previously comments were dropped). timeoutMs is a hard wall-clock budget checked in Lexer.tokenize() and the parser's list loops — the class of defect here is a hang, and a hang must be a loud failure. The parser also throws on a no-progress iteration instead of spinning.

The deparser now decides layout per list: a list is emitted multiline when any statement carries a here-doc body, is backgrounded, or is a comment, and inline (; -joined) otherwise — which is what keeps { echo a; exit 0; } and if …; then …; fi on one line while a here-doc forces a break.

Tests

• __tests__/semantics.test.ts — one case per defect in the issue, asserting the AST shape and, where the script is side-effect free, that bash -c produces identical output for the original and the deparsed form.
• __tests__/corpus.test.ts — the 114 real workflow blocks, committed as __fixtures__/workflows/*.sh, each parsed under a 5s budget and checked for AST equality, deparse idempotence, and bash -n validity.

196 tests pass; eslint, tsc --noEmit and makage build are clean.

Closes constructive-io/constructive-planning#1646

Link to Devin session: https://app.devin.ai/sessions/51c2a0e49ae642a9ab0bbaee1d269b78
Requested by: @pyramation

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring