fix: bound plain-text runs in parse_prompt_attention regex

[fszontagh]

Summary

parse_prompt_attention lexes the prompt with an unbounded plain-text alternative. libstdc++'s std::regex recurses once per matched character, so a prompt of a few tens of kilobytes overflows the stack and segfaults before tokenization.

Bound the run length. Splitting a long run is safe because the function's final pass merges adjacent segments of equal weight, and B is excluded from the character class, so a chunk boundary can never fall inside a BREAK.

Related Issue / Discussion

None.

Additional Information

A 45KB prompt exits 139 (SIGSEGV) during tokenization before the change and completes normally after it.

Parsing results are unchanged: comparing old and new output across weighted parentheses, nested ((...)), [...], escaped \(, multiple BREAKs, BREAKING and bare B, bare colons, the empty string, multi-byte UTF-8, and a 21KB run both bare and inside (...:1.3), every case is byte-identical.

Checklist

• I have read and confirmed this PR follows the contribution guidelines.

[leejet]

The plain-text repetition is bounded now, but the same stack-overflow path still exists in the weight alternative: :([+-]?[.\d]+)\) contains another unbounded +.

With libstdc++ (g++ 14.2, default 8 MiB stack), the PR still segfaults for e.g.:

":" + std::string(30000, '1')

A closing ) is not required: the regex recursively consumes the digit run while attempting the weight alternative before falling back to the literal : alternative.

Could we bound the numeric run as well (to a realistic float length), or avoid unbounded std::regex repetitions in this lexer entirely? We should also make sure oversized/invalid weights cannot terminate through std::stof.

[fszontagh]

Good catch, thanks. Bounded the weight run as well and switched the parse to strtof with a finite check, so an oversized or invalid weight can no longer throw or poison the multipliers.

I had missed it because my equivalence harness ran at -O2, where the smaller frames hide the overflow. At -O0 your ":" + std::string(30000, '1') reproduces exactly as you describe, and is fine after the change.

Parsing is unchanged - old vs new output is byte-identical across weighted parens, nested brackets, escapes, BREAK, bare B, colons, empty input, UTF-8, and long runs both bare and inside (...:1.3).

[leejet]

max_weight_chars = 32 introduces a parsing regression: valid weights longer than 32 characters are treated as normal prompt text instead of weights. Also, please validate the strtof end pointer, otherwise invalid values like . or +. are silently accepted as 0.0.

[fszontagh]

Both fixed. Dropped the weight alternative from the regex and lex the number by hand after a : instead. strtof loops instead of recursing, so there is no stack to blow and no length to cap - max_weight_chars is gone rather than retuned.

The end pointer is checked now, so . and +. fall back to literal text instead of 0.0. Same for 1.2.3, which used to parse as 1.2 through the std::stof prefix parse - flagging it since you did not name that one.

Diffed against the previous revision over 29 targeted cases and ~290k fuzzed prompts: those four are the only differences, the rest is byte-identical. ":" + std::string(30000, '1') survives at -O0 with an 8 MiB stack.