Bash output can corrupt UTF-8 when stdout and stderr interleave

[me2seeks]

Description

The Bash tool can corrupt otherwise valid UTF-8 when stdout and stderr emit at the same time.

ShellTool.run currently calls Stream.decodeText(handle.all), while handle.all is created with Stream.merge(stdout, stderr). This means bytes from two independent pipes are fed through one streaming UTF-8 decoder. If a multibyte character is split across stdout chunks and a stderr chunk arrives in between, the stderr bytes are treated as part of the pending stdout character and replacement characters are emitted.

Each stream is valid when decoded on its own. The corruption only happens because the byte streams are merged before decoding.

I think stdout and stderr should each use their own incremental decoder, with the decoded text events merged afterward. The relative order between separate pipes would still be best-effort, but one stream could no longer corrupt the other stream's decoder state.

I'm happy to submit a fix with regression coverage if this direction makes sense.

Plugins

None

OpenCode version

dev at dd02cea9e7cecd6d5eec6ffcab6741cb51f7f784

Steps to reproduce

1. Ask OpenCode to run this command:

   python3 -c 'import os,time; b="🙂".encode(); os.write(1,b[:1]); time.sleep(.05); os.write(2,b"stderr\n"); time.sleep(.05); os.write(1,b[1:])'

2. Inspect the completed Bash tool output.

3. The output contains Unicode replacement characters even though stdout contains one valid UTF-8 emoji and stderr contains valid ASCII.

The exact ordering of stderr and the emoji is not important. The expected result is that both are preserved without any replacement characters.

Screenshot and/or share link

Not applicable

Operating System

Ubuntu Linux, x86_64

Terminal

Not terminal-specific; this occurs in the subprocess pipe capture path.

[github-actions]

To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.