Documented SFT prompt/completion format is rejected with a KeyError that suggests trainer_type grpo (CR-A03-005) #403

Wave 1 · theme D1-5: Training, merging and export correctness · highest severity High · 2 finding(s) · effort small

Summary

A user follows the manual's SFT example and is told to switch to GRPO. A user who adds a per-row system prompt to preference data trains without it and never finds out.

Primary finding

CR-A03-005 — Documented SFT prompt/completion format is rejected with a KeyError that suggests trainer_type grpo

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end

What happens. _detect_dataset_format classifies any 'prompt' column as the GRPO prompt-only format (data.py:36-41), and process_batch routes prompt/completion rows to _process_user_assistant_format, which raises KeyError recommending grpo (data.py:434-450). The user manual documents {prompt, completion} as the simplest SFT format.

Where.

• forgelm/data.py:23-48
• forgelm/data.py:441-450
• forgelm/data.py:477-490
• docs/usermanuals/en/concepts/data-formats.md:17-30
• forgelm/data.py:34-48 (confirmed)
• forgelm/data.py:434-454 (confirmed)
• forgelm/data.py:634-645
• docs/usermanuals/en/concepts/data-formats.md:14
• docs/usermanuals/en/concepts/data-formats.md:25-30
• docs/usermanuals/en/concepts/data-formats.md:65-66

Evidence.

• repro_data.py section 5: a prompt/completion jsonl raised KeyError "... Found: ['prompt', 'completion']. Detected format: prompt-only format. Suggested trainer: grpo".
• t6/pc_test.py on pc.jsonl raises KeyError "Dataset must contain 'User'/'instruction' and 'Assistant'/'output' columns ... Detected format: prompt-only format. Suggested trainer: grpo". In t3/pref_sys.py, a DPO row with system='You are a pirate...' keeps the system column, but prompt_ids = ['Write', 'Ġone', ..., ').'] with no system text.

Recommended fix. Support prompt/completion (and an optional system column) in the SFT processor, or pass it through to TRL's prompt-completion path. When 'completion' is present, stop _detect_dataset_format from suggesting grpo.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

DOC-D3B05-039 — docs/usermanuals/en/training/sft.md: false claim — SFT supports the single-turn 'instructions' format {prompt, completion}, auto-detected from prompt+completion

severity Medium

Where. docs/usermanuals/en/training/sft.md:57-62; docs/usermanuals/en/concepts/data-formats.md:14,20-27,149 · type: false claim

The document says. SFT supports the single-turn 'instructions' format {prompt, completion}, auto-detected from prompt+completion

What the code does. process_batch routes {prompt, completion} to _process_user_assistant_format, which raises KeyError. _detect_dataset_format calls it 'prompt-only format' and suggests grpo. The run exits 2.

Code references. forgelm/data.py:40-41, forgelm/data.py:442-450, forgelm/data.py:483-488

User impact. The first dataset shape the SFT page shows fails after the model loads, and the message suggests the wrong trainer

Proposed correction. Either support prompt/completion in the SFT processor (or pass it to TRL's prompt-completion path) or remove it from both pages

Turkish mirror. the Turkish page has the same error; fix both in the same PR

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}