fix: preserve Transformers training in safe mode

[Yunnglin]

PR type

• Bug Fix
• New Feature
• Document Updates
• More Models or Datasets Support

PR information

With TWINKLE_TRUST_REMOTE_CODE=0 in Ray mode, Transformers training and forward-only calls fail when a worker passes its own callable model to InputProcessor. The remote decorator rejects these internal arguments before selecting worker-local execution.

Move the processor pipeline into an undecorated process_local method and call it directly from Transformers forward and forward_only. The public __call__ delegates to the same pipeline through its existing remote decorator, retaining safe-mode checks on both dispatch and worker execution. The local method documents its worker-only purpose, rejects remote handles and is not remotely exported. The remote-call framework and deployment configuration are unchanged.

Refs #298 (the Transformers safe-mode training failure).

Experiment results

• The baseline reproduced the exception in Tinker training, normal training, micro-batching and forward-only inference with Ray mode and trust disabled, including real Ray worker processes.
• 50 tests passed with no failures, errors or skips: safe-mode regressions, existing micro-batch and processor tests, and Ray dispatch timeout tests. Coverage includes CPU loss/backward, public callable/type rejection, remote-handle rejection and the local entry point's export boundary.
• All 24 CPU-fixture scenarios passed, including 8 in real Ray worker processes. Loss and gradient norms matched trust-enabled controls exactly.
• A separate real Ray actor probe confirmed that a worker with trust disabled rejects an incoming callable even when the sending handle has trust enabled.
• Pre-commit checks and git diff --check passed. Validation does not cover a full HTTP deployment or GPU/PPU training.

[Copilot]

🟢 Approval recommended

The scoped bypass addresses the reported failure while retaining remote-handle rejection and public safety validation.

0 open findings

What changed in this PR

Fixes Transformers training and inference in Ray safe mode by allowing worker-local processor execution without weakening public remote-call checks.

Changes:

• Adds scoped worker-local invocation for processor instances.
• Uses the local processor path in Transformers forward and forward_only.
• Adds safe-mode, training, custom-processor, cleanup, and remote-handle regression tests.

File	Description
src/​twinkle/​infra/​__init__.py	Adds context-scoped local execution support.
src/​twinkle/​processor/​base.py	Adds the private local processing entry point.
src/​twinkle/​model/​transformers/​transformers.py	Routes Transformers processing through the local entry point.
tests/​model/​test_safe_mode.py	Covers safe-mode behavior and execution boundaries.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.