fix: parse reasoning when the chat template pre-opens <think> (#108)

[solderzzc]

Fixes #108.

Root cause

Qwen3/3.5/3.6 chat templates append <think>\n to the generation prompt when enable_thinking is true. From the cached Qwen3.6-35B-A3B tokenizer_config.json:

{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking is false %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- else %}
        {{- '<think>\n' }}          {# prompt ends inside the block #}
    {%- endif %}

The model therefore emits reasoning with no opening tag of its own, terminated by </think>. But ThinkingStateTracker always started in .responding (Server.swift:1695) and extractThinkingBlock required both tags (Server.swift:2137). So the whole reasoning section was misfiled as content, with a stray </think> inside it.

That is exactly the reported symptom — "the client doesn't get the beginning <think> tag" — and it explains the follow-on behaviour with tool calls: the replayed assistant turn starts mid-thought, so the model learns to omit its own first line.

The reporter noting that vllm-mlx behaves identically fits: it's the template, not MLX.

Changes

• Detect the pre-opened block at runtime — decode the rendered prompt tail (32 tokens) and compare the last opening tag against the last closing tag. This is deliberately not keyed to a model family: the same Qwen line differs by version (cached Qwen3-1.7B does not pre-open, Qwen3.6 does), so hard-coding would break on the next release.
• ThinkingStateTracker(startInThinking:) — starts the parser inside the block; extractThinkingBlock(from:alreadyOpen:) does the same for the non-streaming path.
• Drain the tracker at end of stream — a second, independent defect: a response whose final characters look like the start of a tag stayed in the buffer and was dropped. The stream loop had no flush at finish.
• Tag spellings are now shared constants, and matching picks the earliest match instead of preferring <thinking> found later in the buffer.

Tests

New tests/SwiftLMTests/ThinkingPreOpenedTests.swift — 18 tests over prompt detection (both template branches, prior closed blocks in history, Gemma channel markers), streaming with a pre-opened block (including a closing tag split across chunks), the end-of-stream flush, and non-streaming extraction. testExtractPreOpenedBlockWithoutFlagIsUnchanged pins the old broken output so the difference is explicit.

Verified red: with startInThinking ignored and flush() stubbed to return empty, 9 of the 18 fail.

End-to-end, against mlx-community/Qwen3.5-4B-4bit — a real, unmodified model whose chat_template.jinja carries the same pre-opening form as Qwen3.6:

{%- if enable_thinking is defined and enable_thinking is false %}
    {{- '<think>\n\n</think>\n\n' }}
{%- else %}
    {{- '<think>\n' }}
{%- endif %}

case	result
--thinking, non-streaming	detection fires; reasoning → reasoning_content (461 chars), content = "Tokyo", no leaked tags
--thinking, streaming	148 chunks; reasoning → reasoning_content deltas, content = "\n\n54", no leaked tags
no --thinking (control)	template emits an already-closed block, detection stays off, reasoning_content = null, content = "Tokyo"

Also confirmed on mlx-community/Qwen3.5-0.8B-MLX-4bit that detection fires and reasoning is routed correctly, though that model loops without ever emitting </think>, so it never reaches the content phase.

Full suite: 114 tests across 9 suites, 0 failures. PromptCacheTests aborts under swift test with Failed to load the default metallib — pre-existing at HEAD and environmental, so suites were run individually.

Note on scope

Streaming does not trim the newline that follows </think>, so content begins with \n\n (visible in the table above). That is pre-existing behaviour shared with the self-opened path — the non-streaming path does trim — and this PR leaves it alone rather than widening scope. Worth a follow-up if the inconsistency matters.

🤖 Generated with Claude Code

[solderzzc]

Update: the tool-call flow from the report is now reproduced and verified, not just inferred.

Same model, same flags, two release binaries differing only in Server.swift. A two-round exchange mirroring a real client: round 1 asks a question with a tool available, round 2 replays the assistant turn plus the tool result.

Before (main):

[round1] reasoning_content: None
[round1] content: 'The user is asking for the weather in Paris and wants me to use the tool. I have access to…'
[round1] tool_calls: ['get_weather({"city":"Paris"})']
[round1] LEAKED TAGS IN CONTENT: ['</think>']
[round2] content: 'The tool returned the weather information for Paris. The temperature is 18°C and…'
[round2] LEAKED TAGS IN CONTENT: ['</think>']

After (this branch):

[round1] reasoning_content: 226 chars
[round1] content: None
[round1] tool_calls: ['get_weather({"city":"Paris"})']
[round1] LEAKED TAGS IN CONTENT: none
[round2] reasoning_content: 121 chars
[round2] content: 'The weather in Paris is currently 18°C with light rain.'
[round2] LEAKED TAGS IN CONTENT: none

This is exactly @coolaj86's description: the client receives the model's private reasoning as content with a stray </think> in it, and that malformed turn is what gets replayed into history on the next round. Note the tool call itself succeeds in both — matching the report that tool calls "work" while the surrounding text is wrong.

On the reporter's exact model. I downloaded mlx-community/Qwen3.6-27B-OptiQ-4bit to test against it directly, and could not: it emits garbage on current main regardless of the thinking flag.

--thinking:                    " (' tml_htmlnsub\x00\n\n#\n\n# &г.� c#厉 code"
enable_thinking=false:         "},-idthlaf�(aren系_1!"

The same binary produces coherent output on Qwen3.5-4B-4bit, so this is specific to that model rather than the build. Its config.json carries 498 per-layer quantization overrides (optiq_mixed_precision, 4.77 achieved bpw, embed_tokens at 8-bit over a 4-bit default) and the architecture uses linear_attn blocks. mlx-swift-lm does implement per-layer quantization with a language_model. prefix fallback, so the cause needs its own investigation — I have not identified it, and it is unrelated to this PR. Filing separately unless someone already knows the answer.

So the verification above stands on Qwen3.5-4B-4bit, whose chat_template.jinja pre-opens <think> in exactly the same form as the reporter's Qwen3.6 model.