[Goo-Leo]

FlashRT currently has no Apple Silicon inference route. This adds an experimental Pi0.5 BF16 backend using PyTorch MPS, reachable through load_model(..., hardware="apple_mps") or automatic accelerator detection, without a CUDA/HIP extension build.

The frontend, model binding and hardware providers follow the repository's separate ownership paths and reuse the existing Pi0.5 tensor traversal and safetensors conversion. The change also adds the apple dependency extra, CPU prompt embedding, explicit precision/graph rejection, stable API documentation, and an SSD download/real-input example. No CMake, native kernels, serving or training paths change.

Validation

Apple M5 Pro, 24 GiB unified memory; macOS 27.0.1; native arm64 Python 3.12.14; PyTorch 2.14.1 with MPS available.

.venv/bin/python -m pytest \
  tests/test_apple_backend.py tests/test_pi05_torch_pipeline.py \
  tests/test_npu_contracts.py tests/test_amd_arch_gate.py \
  tests/test_amd_no_global_patching.py tests/test_pi05_state_prompt.py \
  tests/test_pi05_prompt_tokenization.py -q

• 114 passed, 7 skipped for unavailable optional dependencies/tokenizer assets. Coverage includes accelerator precedence, unsupported model/precision options, native-extension isolation, CPU prompt placement, real attention head dimensions, shared-pipeline parity and the public inference wrapper.
• Calibration helpers: 18 passed. Install/import smoke checks: 2 passed; the CUDA binary-presence check was excluded on this MPS-only host.
• tests/test_load_model_use_fp8_kwarg.py: 51 passed and 21 failures caused by absent CUDA runtime/extensions. The unmodified 81904ca baseline reproduces the exact same 21 failing cases in the same environment; no additional failing cases.
• Lazy root import and git diff --check pass. Weights, environments, download caches and generated run artifacts are excluded from the commit.

Real-checkpoint smoke run

Checkpoint: Fisher-Wang/pi05-libero-pytorch, revision ab107fbe6661e8abfec0f13dc27bfd3fafe24c69, a PyTorch conversion of OpenPI's Pi0.5 LIBERO weights with quantile normalization statistics. All five downloaded remote files passed hf cache verify; the tokenizer passed its expected MD5.

.venv/bin/python examples/apple/pi05.py \
  --checkpoint /Volumes/SSD/FlashRT/models/pi05-libero-pytorch \
  --tokenizer /Volumes/SSD/FlashRT/tokenizers/paligemma_tokenizer.model \
  --output /Volumes/SSD/FlashRT/runs/apple_mps_pi05_20261019 \
  --steps 10 --warmup 2 --iters 5 --samples 8 --cpu-reference

Two camera views, ten denoising steps, ten-action horizon, and 147–150 state-prompt tokens:

Measurement	Result
Prewarmed inference including image preprocessing/copy and action export	p50 310.67 ms, p95 311.45 ms
State normalization + prompt update + inference, eight observations	p50 338.56 ms
MPS tensor allocations	5.38 GiB
Fixed-noise repeated output, max absolute difference	0
CPU comparison, seven raw robot channels, one observation	cosine 0.99999737, max abs 0.00390625
CPU comparison, seven unnormalized robot channels, one observation	cosine 0.99999818, mean abs 0.00034081

All eight real camera/state observations produce finite (10, 7) actions, and changing observations changes the output. A final smoke run after organizing the frontend/model/hardware modules also passed with identical numerical comparison metrics.

Scope and limitations

This is BF16 eager execution with reusable buffers, time-modulation precomputation and SDPA. It does not implement torch.compile, graph capture/replay, custom Metal kernels, MLX, FP8/FP4, temporal frame caching or other model families. The latency numbers are short local smoke measurements, not production statistics or CUDA graph replay comparisons.

The CPU check shares the model traversal and converted BF16 weights; CPU GEMM accumulates exact BF16 operand values in FP32 and rounds to BF16 before bias addition. It checks device/provider numerics, not independent OpenPI accuracy or LIBERO simulator task success. CPU embedding storage and runtime overhead are additional to the reported MPS allocation.

[LiangSu8899]

Thanks for the great work and the expanded support!!

[Goo-Leo]

Enjoy and have fun~