[Liuchenbing-2026]

PR type

• Bug Fix
• New Feature
• Document Updates
• More Models or Datasets Support

PR information

Adapter checkpoint collection currently gathers parameters only. When a PEFT modules_to_save module has persistent buffers, such as candidate token IDs or calibration temperatures, these are omitted from adapter_model.safetensors; PEFT then raises a missing-key error when restoring the checkpoint.

Collect persistent buffers from the selected adapter's saved modules before applying the existing PEFT key normalization. Nonpersistent buffers, the original module, other adapters, and backbone buffers remain excluded. The change does not materialize the full backbone state dictionary.

AI assistance was used for implementation and testing.

Experiment results

• python -m pytest tests/transformers/test_adapter_persistent_buffers.py -q --confcutdir=tests/transformers: 1 passed.
• The same regression with the upstream checkpoint collection method fails because base_model.model.head.temperature is missing.
• The regression serializes and reloads a real PEFT adapter, verifies restoration of float and integer buffers, and checks exclusion of scratch buffers, backbone buffers and a second adapter.
• pre-commit run --all-files: passed.
• Validation used Torch 2.10.0 and PEFT 0.19.0. Multi-rank FSDP/EP validation has not been performed for this change.

[Liuchenbing-2026]

@tastelikefeet could you please review this adapter-checkpoint correctness fix?

I rechecked the committed regression with the exact _get_adapter_state_dict_for_save method from the original base (ec8fd1f0660b09ed60938d02d2439533a5aa5c65), current main (9953ea1ebbbdbe59c3831702157d3c147ca0ee26), and this PR's unchanged head (12b5cff4e2d97f5070a873f78fcc3b98dc42f72c):

Checkpoint collection method	Result
Original base	Fails with KeyError: base_model.model.head.temperature
Current main	Same missing-buffer failure
PR head	1 passed, no skips

This is a focused CPU regression using real PEFT modules and safetensors save/load, with a small strategy fixture for parameter collection. It checks persistent float/integer buffer inclusion, excludes nonpersistent/backbone/other-adapter buffers, and restores a changed temperature buffer through PEFT. The harness substitutes the exact method from each revision into the same runtime; this is not a full main-branch training-suite run.

Rechecked on Linux/aarch64, Python 3.12.13, Torch 2.10.0 and PEFT 0.19.0. An initial unprivileged attempt stopped at the separate import-time lock-directory permission issue; the results above were rerun in a writable working directory. Multi-rank FSDP/EP is still outside this validation scope.

This is an independently reproduced bug found during adapter round-trip testing, rather than a claim to resolve an existing issue number. Thanks!

[tastelikefeet]

Let me see

[tastelikefeet]

Thanks! I will merge this PR into the main branch.
The design of this piece of code is not good enough, I will do a small refactor to fix this problem.