simpo

Full post-training pipeline for Qwen2.5-1.5B — SFT → SimPO → GRPO on free T4/P100 GPUs. GSM8K accuracy jumps from 23% (base) to 61% (GRPO) using Unsloth 4-bit LoRA, TRL, and HuggingFace Hub checkpointing.

Headless LLM fine-tuning in 3 lines — smart defaults, VRAM-aware batch sizing, multi-run SLAO, GGUF export for Ollama.

End-to-end RLHF pipeline with reward debiasing, DPO vs SimPO comparison, and statistical significance testing on Anthropic hh-rlhf dataset.

LocalSight · 198M-A64M 思考型 MoE LLM，2×RTX4090 从零训练（pretrain→SFT→SimPO→RLAIF→Agent RL），GGUF/Ollama 可运行

Domain-specific benchmark for B2B sales agents — 250 tasks, SimPO judge model, published on HuggingFace.

Trustworthy QA alignment experiments with citation, abstention, and from-scratch DPO, SimPO, and ORPO losses.

Official code and reproducibility artifacts for Reasoning on a Spectrum (COLM 2026)

RLHF experiments with Unsloth: DPO, ORPO, SimPO, KTO. Training scripts, notebooks, and quick evaluation utilities.