feat: add native typed decision training components

README.md

@@ -406,6 +406,7 @@ foundation for building customizable, enterprise-grade training services.
 | Component Type | Component Link                                                                                           | Component Function                                                                      | Author              |
 | -------------- | -------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | ------------------- |
 | Patch          | [qwen3_moe_transformers4_patch](https://www.modelscope.cn/models/twinkle-kit/qwen3_moe_transformers4_patch) | Fixes Qwen3 MoE model hang issue during FSDP2 training, effective for transformers==4.x | ModelScope Official |
+| Decision | [Typed decision training](cookbook/transformers/decision/README.md) | Candidate scoring, soft-label losses, LoRA head persistence and question accuracy | Community |
 
 ## Contributions
 

README_ZH.md

@@ -389,6 +389,7 @@ for epoch in range(3):
 | 组件类型 | 组件链接                                                                                           | 组件功能                                                                      | 作者              |
 | -------- | -------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ----------------- |
 | Patch    | [qwen3_moe_transformers4_patch](https://www.modelscope.cn/models/twinkle-kit/qwen3_moe_transformers4_patch) | 修复 Qwen3 MoE 模型在 FSDP2 训练期间挂起的问题，适用于 transformers==4.x | ModelScope 官方 |
+| Decision | [决策模型训练](cookbook/transformers/decision/README.md) | 候选打分、软标签损失、LoRA 打分头保存与问题级精度 | 社区 |
 
 ## 开源贡献
 

cookbook/transformers/decision/README.md

@@ -0,0 +1,136 @@
+# Typed decision training
+
+Twinkle can train and score typed decisions through its existing TransformersModel, InputProcessor,
+Loss and PEFT interfaces. Predictions are probabilities over each question's legal options;
+there is no autoregressive answer-token generation. A probability is not automatically calibrated:
+retain the checkpoint's calibration settings, or fit new settings on a disjoint calibration split.
+
+## 1. Model and task
+
+| Family | Execution | Head | Loss preset |
+| --- | --- | --- | --- |
+| generic CausalLM | One question per prompt | Tokenizer-derived verbalizer | scoring |
+| JEV | One question per prompt; text/image input | Released verbalizer and bias | distill_rps |
+| Clef | All questions in a record jointly; text input | Span pooling and joint schema decoder | smooth_brier |
+| OmniJev | Cached prefix and separate option branches; text/image/precomposed video | FiLM and ordinal head | ce_rps |
+| StartLux | One question per prompt; text input | Last-position FP32 A-Z projection | scoring |
+
+`noul`, `choice`, and `score` use different option orders across families. The preprocessor preserves
+those orders and remaps targets. Do not substitute one family's prompt or calibration for another.
+Generic/JEV verbalizers require distinct single-token symbols and allow up to 16 choice options.
+StartLux allows 2–26 options; wide choices must be explicitly decomposed outside this recipe.
+
+## 2. Validation and scope
+
+| Check | Evidence / scope |
+| --- | --- |
+| Loss and gradients | Four presets compared against independent unpadded formulas at 1e-12 |
+| Accumulation | Uneven question counts match the full-batch normalized gradient |
+| LoRA persistence | Tiny CausalLM updates both adapter and head; save/reload logits match exactly on CPU |
+| Head contracts | Clef joint spans/padding, OmniJev FiLM/ordinal gradients, JEV image payload |
+| Regression suite | Decision loss, encoding, augmentation and checkpoint regression tests; full repository pre-commit checks |
+| Native training API | Tiny model forward/backward, optimizer step, save, resume from step 3 to 6 and reload; StartLux-4B two-step training, save and adapter reload/scoring on one Ascend NPU |
+| Released-model quality | Full checkpoint-specific accuracy and visual benchmark validation pending |
+
+Tests use synthetic fixtures and do not establish released-checkpoint quality. Clef image span
+alignment, packing, sequence/context parallelism and multi-adapter serving are not supported here.
+The example uses one device; multi-rank training requires separate sharding and checkpoint validation.
+
+## 3. Installation and reproducibility
+
+Develop from a checkout and install its declared dependencies without changing repository bounds:
+
+```bash
+python -m pip install -e . pytest
+python -m pytest tests/decision -q --confcutdir=tests/decision
+pre-commit run --all-files
+```
+
+An Ascend validation environment uses the base image below (no image export required), Python 3.12,
+Torch 2.10.0, torch-npu 2.10.0.post4, CANN 9.1.0, Transformers 5.14.1 and PEFT 0.19.0.
+Use a new container, mount only assigned devices and a task directory, and install Twinkle into an
+isolated virtual environment. The image also contains unrelated inference packages; installing
+training dependencies does not validate those packages' inference environments.
+
+```text
+quay.io/ascend/vllm-ascend@sha256:8a397e82a22f55de25ae355a957b91e843a16ce704a03badc0a8e9915f68d84f
+```
+
+If the container exposes one physical NPU, its runtime-visible ordinal is `0`, even when its host
+card number differs. Record both device identities in private deployment notes.
+
+## 4. Train, resume and score
+
+No model or dataset downloads are needed for this synthetic smoke test; dependencies must already
+be installed. Use a fresh output directory for every run.
+
+```bash
+python cookbook/transformers/decision/make_tiny.py --output /tmp/decision-tiny
+python cookbook/transformers/decision/decision.py train \
+  --model /tmp/decision-tiny --data /tmp/decision-tiny/mini.jsonl \
+  --output /tmp/decision-train --steps 3 --dtype float32
+python cookbook/transformers/decision/decision.py infer \
+  --model /tmp/decision-tiny --adapter /tmp/decision-train/final \
+  --data /tmp/decision-tiny/mini.jsonl --output /tmp/decision-infer --dtype float32
+python cookbook/transformers/decision/decision.py train \
+  --model /tmp/decision-tiny --resume /tmp/decision-train/final \
+  --data /tmp/decision-tiny/mini.jsonl --output /tmp/decision-resume --steps 6 --dtype float32
+```
+
+`--adapter` initializes weights; `--resume` restores training state. They are mutually exclusive.
+`steps` is the total target number of updates. The recipe records consumed examples with the
+checkpoint; preserve data ordering and augmentation settings when resuming.
+
+| Released family | Arguments to add to the same train/infer command |
+| --- | --- |
+| JEV | `--family jev --model <BASE_CHECKPOINT> --head-dir <JEV_RELEASE> --adapter <JEV_RELEASE>/adapter_vllm` |
+| Clef | `--family clef --model <MERGED_CLEF_CHECKPOINT> --head-dir <CLEF_HEAD_DIRECTORY>` |
+| OmniJev | `--family omnijev --model <BASE_CHECKPOINT> --head-dir <OMNIJEV_RELEASE> --adapter <OMNIJEV_ADAPTER>` |
+| StartLux | `--family startlux --model <STARTLUX_CHECKPOINT>` |
+
+For JEV/OmniJev, keep the released adapter: omitting it discards the released decision capability.
+Use the matching tokenizer, head artifacts and calibration. StartLux also requires the official
+`startlux_decision` Python package on PYTHONPATH. The validated source revision is
+`StartLuxLabs/StartLux-Decision@0e7a2e81b9c92756e26d8edd843a44d50e362669`.
+For StartLux 4B/35B, select language attention targets and exclude the vision tower and vocabulary
+head; do not claim training benefit from a smoke test.
+
+Training accepts hard labels or finite nonnegative distributions summing to one. Targets are never
+rendered into model inputs. `--shuffle-fields`, `--drop-instructions` and `--subsample-options`
+are explicit training-only augmentation options for generic/JEV/Clef/OmniJev; StartLux rejects these flags. Disable augmentation for evaluation.
+Inference writes `predictions.jsonl` and `metrics.json` with per-question `acc` and per-encoded-record
+`record_acc`. A JEV/OmniJev source record is expanded before this accounting.
+
+Open-Jev is an explicit online download, separate from the offline smoke dataset:
+
+```bash
+python cookbook/transformers/decision/fetch_open_jev.py \
+  --dataset ZefanCai/Open-Jev --revision <DATASET_REVISION> \
+  --subset release-v2-redistributable --split train --output /tmp/open-jev-train.jsonl
+# Use --open-jev-format --data /tmp/open-jev-train.jsonl with train or infer.
+```
+
+Respect the dataset's license and preserve separate training, validation, calibration and test splits.
+Image rows accept local images. OmniJev videos must first be converted to a frame mosaic with
+`twinkle.model.decision.omnijev_media.video_state`; this needs ffmpeg/ffprobe. Passing raw video paths
+into the training encoder raises an error.
+
+## 5. Acceptance before publication
+
+| Gate | Required evidence |
+| --- | --- |
+| Input | Exact prompt IDs, option order, target exclusion, image grids and branch spans |
+| Numeric | Head logits/probabilities and loss/gradient comparisons against each released reference |
+| Training | Finite updates, adapter/head changes, frozen-backbone audit, checkpoint recovery |
+| Quality | Independent held-out scoring; report each suite and its denominator |
+| Performance | Same device, precision, input and warmup; keep performance artifacts local |
+
+The scoring head is saved through PEFT `modules_to_save`; vocabulary embeddings are weak references
+and are not duplicated into head state. FP32 activation must happen after adapter creation and before
+the optimizer. Source attribution is retained in reused numerical modules.
+
+## 6. Summary
+
+Use the common Twinkle training API while retaining each model's decision format and head semantics.
+Passing the included contract tests is an engineering milestone; it does not replace full released
+model accuracy, multimodal, distributed or performance acceptance.

cookbook/transformers/decision/decision.py

@@ -0,0 +1,152 @@
+# Copyright (c) ModelScope Contributors. All rights reserved.
+"""Train and evaluate typed decisions through Twinkle's Transformers backend."""
+import argparse
+import json
+import math
+from pathlib import Path
+
+import torch
+from transformers import AutoProcessor, AutoTokenizer
+
+import twinkle
+from twinkle.loss.decision import DecisionLoss
+from twinkle.metric.decision import DecisionAccuracy
+from twinkle.model import TransformersModel
+from twinkle.model.decision.encoding import DecisionEncoder
+from twinkle.model.decision.model import activate_decision_head, attach_decision_head, decision_lora_config, initialize_missing_adapter_head
+from twinkle.preprocessor.decision import (ClefPreprocessor, DecisionAugmentConfig, JevPreprocessor,
+                                           OmniJevPreprocessor, OpenJevPreprocessor, StartLuxPreprocessor)
+from twinkle.processor.decision import DecisionInputProcessor
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('action', choices=['train', 'infer'])
+    parser.add_argument('--model', required=True, help='Local base or merged checkpoint directory')
+    parser.add_argument('--family', choices=['generic', 'jev', 'clef', 'omnijev', 'startlux'], default='generic')
+    parser.add_argument('--head-dir', help='Original decision head artifacts, required for released models')
+    parser.add_argument('--adapter', help='Released or trained PEFT adapter directory')
+    parser.add_argument('--resume', help='Resume optimizer and model state from a training checkpoint')
+    parser.add_argument('--data', required=True, help='Local JSONL; use disjoint files for training and testing')
+    parser.add_argument('--open-jev-format', action='store_true')
+    parser.add_argument('--output', required=True)
+    parser.add_argument('--loss', choices=['scoring', 'distill_rps', 'smooth_brier', 'ce_rps'])
+    parser.add_argument('--steps', type=int, default=10)
+    parser.add_argument('--lr', type=float, default=1e-4)
+    parser.add_argument('--max-length', type=int, default=8192)
+    parser.add_argument('--lora-targets', nargs='+', default=['q_proj', 'v_proj'])
+    parser.add_argument('--dtype', choices=['float32', 'bfloat16'], default='bfloat16')
+    parser.add_argument('--shuffle-fields', action='store_true')
+    parser.add_argument('--drop-instructions', type=float, default=0)
+    parser.add_argument('--subsample-options', type=int, default=0)
+    args = parser.parse_args()
+    if args.resume and (args.action != 'train' or args.adapter):
+        raise ValueError('Use --resume for training recovery; --adapter is a separate weights-only initialization')
+    if args.family == 'startlux' and (args.shuffle_fields or args.drop_instructions or args.subsample_options):
+        raise ValueError('StartLux augmentation is not supported; preserve the official renderer')
+    output = Path(args.output)
+    if output.exists():
+        raise FileExistsError('Choose a new output directory')
+    if args.steps < 1:
+        raise ValueError('steps must be positive')
+    twinkle.initialize(mode='local')
+    if args.family in ('omnijev', 'jev'):
+        processor = AutoProcessor.from_pretrained(args.model)
+    else:
+        processor = AutoTokenizer.from_pretrained(args.model)
+    tokenizer = getattr(processor, 'tokenizer', processor)
+    if tokenizer.pad_token_id is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    if tokenizer.pad_token_id is None:
+        raise ValueError('Tokenizer needs a pad or EOS token')
+    wrapper = TransformersModel(model_id=args.model, dtype=getattr(torch, args.dtype),
+                                mixed_precision='no' if args.dtype == 'float32' else 'bf16',
+                                attn_implementation='sdpa')
+    attach_decision_head(wrapper.model, processor, args.family, args.head_dir or args.model)
+    if args.adapter:
+        from peft import PeftConfig, set_peft_model_state_dict
+        from peft.utils.save_and_load import load_peft_weights
+        adapter = PeftConfig.from_pretrained(args.adapter)
+        adapter.inference_mode = args.action != 'train'
+        factory_head = 'scoring_head' not in (adapter.modules_to_save or [])
+        adapter.modules_to_save = list(dict.fromkeys(list(adapter.modules_to_save or []) + ['scoring_head']))
+        wrapper.add_adapter_to_model('decision', adapter)
+        state = load_peft_weights(args.adapter, device='cpu')
+        if factory_head:
+            state = initialize_missing_adapter_head(wrapper.model, state, 'decision')
+        restored = set_peft_model_state_dict(wrapper.model, state, adapter_name='decision')
+        missing_lora = [key for key in restored.missing_keys if 'lora_' in key]
+        if missing_lora:
+            raise ValueError(f'Missing adapter weights: {missing_lora}')
+        if restored.unexpected_keys:
+            raise ValueError(f'Unexpected adapter keys: {restored.unexpected_keys}')
+    else:
+        wrapper.add_adapter_to_model('decision', decision_lora_config(
+            r=8, lora_alpha=16, target_modules=args.lora_targets))
+    activate_decision_head(wrapper.model)
+    wrapper._default_tokenizer = processor
+    wrapper.set_processor(DecisionInputProcessor, pad_token_id=tokenizer.pad_token_id,
+                          padding_side='right' if args.action == 'train' else 'left')
+    mode = args.loss or {'generic': 'scoring', 'jev': 'distill_rps', 'clef': 'smooth_brier',
+                        'omnijev': 'ce_rps', 'startlux': 'scoring'}[args.family]
+    wrapper.set_loss(DecisionLoss, mode=mode)
+    if args.action == 'train':
+        wrapper.set_optimizer('AdamW', lr=args.lr, foreach=False)
+    start_step = 0
+    if args.resume:
+        progress = wrapper.resume_from_checkpoint(args.resume, adapter_name='decision')
+        start_step = int(progress['consumed_train_samples'])
+        if start_step >= args.steps:
+            raise ValueError('steps must exceed the already completed step count')
+    # Token-level accuracy is not defined for a distribution-valued label.
+    group = wrapper.optimizer_group['decision']
+    for state in (group.train_status, group.eval_status):
+        state.metrics = [DecisionAccuracy(wrapper.device_mesh, group._dp_group)]
+    config = None
+    if args.action == 'train':
+        config = DecisionAugmentConfig(drop_instructions=args.drop_instructions,
+            subsample_options=args.subsample_options, shuffle_fields=args.shuffle_fields)
+    pre_cls = {'generic': JevPreprocessor, 'jev': JevPreprocessor, 'clef': ClefPreprocessor,
+               'omnijev': OmniJevPreprocessor, 'startlux': StartLuxPreprocessor}[args.family]
+    source = [json.loads(line) for line in Path(args.data).read_text().splitlines() if line.strip()]
+    preprocessor = pre_cls(augment_config=config)
+    if args.open_jev_format:
+        if args.family == 'startlux':
+            raise ValueError('Open-Jev normalization supports generic/JEV/Clef/OmniJev schemas')
+        preprocessor = OpenJevPreprocessor(pre_cls(), augment_config=config)
+    normalized, _ = preprocessor(source)
+    encoder = DecisionEncoder(processor, args.family, args.max_length)
+    rows = [encoder(row) for row in normalized]
+    if not rows:
+        raise ValueError('Dataset has no decision records')
+    output.mkdir(parents=True)
+    (output / 'run.json').write_text(json.dumps(vars(args), indent=2))
+    if args.action == 'train':
+        if any('target_probs' not in row for row in rows):
+            raise ValueError('All training records require targets')
+        for step in range(start_step, args.steps):
+            result = wrapper.forward_backward(inputs=[rows[step % len(rows)]])
+            if not math.isfinite(float(result['loss'])):
+                raise RuntimeError('Non-finite decision loss')
+            norm = wrapper.clip_grad_norm(max_grad_norm=1.)
+            if norm is not None and not math.isfinite(float(norm)):
+                raise RuntimeError('Non-finite decision gradient')
+            wrapper.step(); wrapper.zero_grad()
+            print(json.dumps({'step': step + 1, 'loss': float(result['loss'])}), flush=True)
+        wrapper.save(name='final', output_dir=str(output), adapter_name='decision', save_optimizer=True,
+                     consumed_train_samples=args.steps)
+    else:
+        metric = DecisionAccuracy()
+        with (output / 'predictions.jsonl').open('w') as stream:
+            for row in rows:
+                result = wrapper.forward_only(inputs=[row], return_logits=True)
+                status = group.eval_status
+                metric.accumulate(status.inputs, status.outputs)
+                scores = result['logits'].masked_fill(~result['option_mask'], -torch.inf)
+                probs = scores.float().softmax(-1).cpu().tolist()
+                stream.write(json.dumps({'probabilities': probs}) + '\n')
+        (output / 'metrics.json').write_text(json.dumps(metric.calculate(), indent=2))
+
+
+if __name__ == '__main__':
+    main()