Skip to content

feat: add StartLux decision LoRA training on Ascend - #305

Open
Liuchenbing-2026 wants to merge 6 commits into
modelscope:mainfrom
Liuchenbing-2026:startlux_decision_npu_training
Open

Liuchenbing-2026 wants to merge 6 commits into
modelscope:mainfrom
Liuchenbing-2026:startlux_decision_npu_training

Conversation

@Liuchenbing-2026

@Liuchenbing-2026 Liuchenbing-2026 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

PR type

  • New Feature
  • Bug Fix
  • Document Updates
  • More Models or Datasets Support

PR information

Add a StartLux decision LoRA cookbook using TransformersModel, the existing NPU Qwen3.5 kernels, a custom input processor and candidate-only soft cross-entropy. The original tokenizer/renderer and candidate order are preserved; targets never enter the prompt. Language LoRA parameters are audited and the original backbone, vision and vocabulary head remain frozen. The example saves adapters and training state and provides an independent paired hard-label evaluator.

Two training-path fixes are required for this example: local optimizer restore must not broadcast without a process group, and gradient synchronization must occur after a complete accumulation group. Each has an independent numerical regression test. The README contains the exact environment, training, resume and evaluation commands, including the tested NPU dependency override.

Experiment results

  • python -m unittest discover -s tests/transformers -p test_startlux_decision.py -v: 7 CPU contract tests passed, including structured JSON evidence and language-linear LoRA isolation.
  • python -m pytest tests/transformers/test_native_fsdp_strategy.py tests/transformers/test_accumulation_boundaries.py -q: each three-test file passed separately; optimizer resume and accumulation match their uninterrupted/large-batch numerical references.
  • Changed-file pre-commit checks passed.
  • Real StartLux-4B on Ascend: finite forward/backward and gradient norms, two optimizer updates with accumulation=4, checkpoint save, and full-state resume were exercised.
  • Real StartLux-35B-A3B on two Ascend devices completed forward/backward, finite optimizer updates, checkpoint save and resume. The DataLoader receives a global micro-batch derived from the requested per-rank batch size. Distributed resume is not claimed to be bitwise identical.
  • Full domain training and independent quality comparisons continue. No seven-suite accuracy gain is claimed; private records and measurements are not included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants