[tannevaled]

Closes #48.

Deeply nested source overflowed the Go stack, which is a fatal error: rather than a panic: — recover() never sees it, so neither this package's own recover in Parse nor an embedder's could contain it, and the process died. x = + f(×50000, about 150 KB of source, was enough on v0.11.1.

It matters beyond this repository because go-embedded-ruby/ruby calls Parse as the first statement of its public Run, so a host that merely compiles untrusted Ruby was exposed without ever executing it.

Two things the measurement settled that reading would not have

Where to count — no single function sees every production

Instrumenting four candidates at nesting 1000:

production	parseExprOrAssign	parseTernary	parseRange	parsePrimary
array / hash / paren / call / interp	1001–1002	1001	1001	1001
block (f { … })	1002	1001	1001	1
ternary	2	1001	1	1
!×N	2	1	1	1

parsePrimary is blind to blocks; parseExprOrAssign is blind to ternaries. The four productions that actually overflow — array, call, interpolation, block — all route through parseExprOrAssign, and ternary through parseTernary, so both carry the count and share one counter. !×N recurses through neither and never overflowed (clean at 100 000), so it needs nothing.

The counter measures FRAMES, not brackets — and I got this wrong first

parseExprOrAssign and parseTernary nest within each other, so for every bracket-like production the counter runs at exactly 2× the source nesting. Measured at nesting 2000: array 4001; hash/paren/call/interp/block 4003 each; ternary 2003.

A first revision set the limit to 16384 believing it counted brackets. That capped nesting at 8192 and rejected [×9500 — a shape MRI 4.0.5 accepts. My own verification caught it, because I checked the end where it must keep accepting and not only the end where it must refuse.

A limit that is too low is as much a defect as no limit at all, just a quieter one. TestNestingMriAcceptsIsStillAccepted is what would have caught it, and is in this PR for that reason.

The limit: 32768 frames, between two measured bounds

bound	nesting	counter units
deepest shape MRI accepts ([×N, ruby -c → "Syntax OK"); it raises SystemStackError at 10000, and refuses hash/call/interp/block/ternary by 8000 as a Prism SyntaxError	9 500	19 000
32768	—	32 768
deepest observed safe here (worst productions f(×N and f { ×N are fatal at 50000, safe at 45000; [×N and "#{×N fatal at 55000)	45 000	90 000

1.7× above the first, 2.7× below the second.

It is a frame count rather than a stack-byte budget because Go offers no portable stack-pointer read, and a saved stack address is useless as a baseline: the runtime copies a goroutine's stack when it grows, leaving the value stale by the relocation delta. MRI can budget bytes; we cannot. The honest consequence, recorded in the comment: a production costing far more stack per frame than any probed here would reach the real ceiling first.

Why the decrement is not deferred

p.fail panics, that panic unwinds to Parse — the only recover() in the package — and the Parser is discarded there, so a skipped decrement on the error path cannot be observed, and the hot path pays nothing for a defer.

I checked for an internal recover before relying on that, because speculative parsing would have made the counter leak silently. There is none. TestDepthDoesNotAccumulateAcrossStatements parses 40 000 shallow statements — far more total frames than the limit allows at once — to pin that the ordinary return path does decrement.

Verification

array 9500  -> PARSED-OK          (MRI accepts it, so we must)
array 16000 -> PARSED-OK
array/call/interp/block/ternary at 100000 -> parse error "nesting of 32769 is too deep", exit 0

Before this change those five were fatal error: stack overflow, exit 2.

Full suite green (parser, ast, lexer, token), gofmt -l . empty.

Cost: not resolvable at this sample size, and I will not claim better. Three runs each at -benchtime=3s, against origin/main's parser.go with the same test file, moved in opposite directions — a 16-deep call chain +2.0 %, a 200-method ordinary program −1.6 % — with overlapping ranges in both. Two increments and a compare per expression is not measurable here; that is not the same as being free.

Note for the dependent

go-embedded-ruby/ruby pins v0.11.1, so it does not get this until a tagged release and a go.mod bump. go-embedded-ruby/ruby#768 tracks the VM-side sibling of this defect and references #48; worth saying there which release carries the parser fix.

🤖 Generated with Claude Code

[tannevaled]

Red on the first push, for two reasons — both mine, both now fixed

1. The test timed out all three OS lanes at 600s

CI runs go test -race -coverpkg=... -coverprofile=... with go test's 10-minute default and no explicit -timeout. I had measured 2.8s locally with a plain go test and pushed on that. Under -race alone the same test takes 33s locally; with coverage instrumentation on a slower runner it never finished.

The depth is now 20000 rather than 100000 — already past the limit for every bracket production, so a deeper input proves nothing more and costs real time. Under CI's exact command the package now takes 63s.

The depths are now per production, because one number was wrong for one of them: the limit counts frames, and a ternary chain runs at ratio 1.0 where bracket productions run at 2.0. So 20000 levels of ternary is 20001 frames — under the 32768 ceiling, and that subtest silently stopped testing anything. The suite caught it on the next run.

2. The coverage gate named a check I nearly deleted as dead code

The gate reported the failNestingTooDeep() inside parseExprOrAssign as uncovered. The tempting reading was dead code, and the evidence looked strong: eleven shapes — array, hash, paren, call, interp, block, ternary, range, not-paren, array-ternary, splat — all cross the limit inside parseTernary, never there.

The answer to "can this happen?" is yes, by a production none of those eleven covers:

assign-chain n=20000   bytes=80006     parsed OK                              trip=(did not trip)
assign-chain n=60000   bytes=240006    ERR: nesting of 32769 is too deep      trip=parseExprOrAssign
assign-chain n=120000  bytes=480006    ERR: nesting of 32769 is too deep      trip=parseExprOrAssign
assign-chain n=300000  bytes=1200006   ERR: nesting of 32769 is too deep      trip=parseExprOrAssign

An assignment chain (y = *N) recurses through parseExprOrAssign alone, at ratio 1.0. Deleting that statement would have left it with no guard at all — at 300000 it is 1.2 MB of source.

So the gate was reporting a missing test, not dead code. The assignment chain is now a row in the table with that reason beside it, and the comment in parser.go no longer claims only parseTernary matters. Both check sites are load-bearing and that is now measured rather than assumed.

Verified under CI's exact invocation this time, gate included

go test -race -coverpkg=<CI's, /ast excluded> -coverprofile=... ./...   ok, 63s
.github/coverage-gate.sh cover.out                                     3472 of 3472 statements covered, exit 0
gofmt -l .                                                             empty

The PR body's claim that "the four productions that actually overflow all route through parseExprOrAssign, and ternary through parseTernary" was incomplete — assignment chains route through parseExprOrAssign only. That is corrected in the code comment; treat this comment as the amendment to the body.

🤖 Generated with Claude Code