feat: prefetch streamed layers during compute

[assouan]

Summary

This PR adds asynchronous lookahead to --stream-layers. While the current
parameter-bearing graph-cut segment is computing, parameters for future segments
can be copied from CPU memory through a separate transfer backend or queue when
supported. This changes layer streaming from a serial load-then-compute path into
a rolling pipeline that can hide parameter-transfer latency behind useful GPU
work.

--layer-prefetch-depth <N> controls that pipeline. Its default 0 preserves
the existing synchronous behavior, 1 overlaps the next segment with the active
segment, and larger values request deeper lookahead. The effective depth is
reduced automatically when it does not fit: prefetch never forces a positive
--max-vram budget to be exceeded, and the active segment is never evicted.
When asynchronous transfer is unavailable or a runtime prefetch allocation
fails, execution safely continues with synchronous streaming.

The implementation also adds --resident-layers <N|auto> as a maximum, rather
than a required count. This control is important for the prefetch pipeline:
--resident-layers 0 --layer-prefetch-depth 1 disables optional residency and,
when the prefetch fits the VRAM budget, maintains the minimal rolling window of
one active segment and one segment loading ahead. auto (the default, represented
by -1) retains as many leading segments as the remaining budget permits, while
a positive value caps that resident prefix.

The streaming path additionally:

• gives the requested prefetch window priority over optional resident segments;
• accounts for canonical backend allocation sizes and deduplicates parameters
  shared by resident, active, and prefetched segments;
• keeps cross-segment shared parameters on the safe synchronous path;
• reuses prefetched allocations when a segment becomes active and can retain the
  same allocation if that segment is resident;
• evicts residents from the end of the resident prefix when runtime memory
  pressure would otherwise prevent prefetching;
• releases stale residents and prefetches when the graph plan changes, and does
  not retain one-shot preparation graphs across the sampling loop;
• preserves the existing C API layout by exposing the controls through the
  versioned sd_layer_stream_params_t extension and
  new_sd_ctx_with_layer_stream();
• exposes the controls consistently through the CLI and server and documents
  their VRAM-budget semantics.

Related Issue / Discussion

• Reimplements the asynchronous-prefetch goal of #1626 on the current runner weight-staging architecture. That PR overlapped the next segment's H2D transfer with the current segment's compute, but its author closed it after #1644 removed the partial-buffer machinery it depended on and required the idea to be ported to RunnerWeightManager.
• Builds on the resident-layer control discussion in #1830. In this PR, residency is a VRAM-bounded maximum and also provides the explicit zero-resident mode needed for the minimal active-plus-prefetch pipeline.

Additional Information

Effective behavior

Configuration	Behavior
--layer-prefetch-depth 0	Existing synchronous layer streaming
--layer-prefetch-depth 1	Prefetch the next segment while the active segment computes, when it fits
--layer-prefetch-depth N	Prefetch up to N future segments, reduced to fit the VRAM budget
--resident-layers 0 --layer-prefetch-depth 1	No optional residents; one active plus one prefetched segment when budget permits
--resident-layers auto or -1	Use the remaining budget for an automatically selected resident prefix
--resident-layers N	Keep at most N leading parameter-bearing segments resident
--max-vram 0	Graph-cut segmentation remains disabled, so residency and prefetch controls have no effect

Both controls only apply when --stream-layers is enabled and the diffusion
parameters use the CPU backend.

Verification

• CPU Release build completed successfully (220/220 compile and link steps),
  including the CLI and server.
• CUDA Release build completed successfully for the shared library and CLI
  (13/13 incremental steps); the CUDA server target also built successfully.
• Focused tests passed for:
  • VRAM-budget, resident-limit, and prefetch-depth policy boundaries;
  • physical staging-buffer release across partial tensor lifetimes;
  • resident eviction and stale-prefetch cleanup after graph-plan changes;
  • current and undersized versioned C API structures;
  • compatibility of an executable built against the previous public API.
• CUDA integration was exercised with MiniMax-H3 at 256x256, 22 frames, two
  sampling steps, and seed 42:
  • --max-vram cuda1=2 --resident-layers 0 --layer-prefetch-depth 1 selected
    zero residents and depth one; all 102 queued prefetches were activated.
  • --max-vram cuda1=8 --resident-layers 50 --layer-prefetch-depth 10 selected
    22 residents and depth ten; all 80 queued prefetches were activated.
  • Both configurations generated all 22 frames without warnings or transfer
    failures, and all 22 output SHA-256 hashes matched their reference runs.
• git diff --check passes, and targeted clang-format verification reports no
  remaining replacements in the modified C/C++ lines.

CUDA hardware was available for integration testing. Vulkan, ROCm, and Metal
were not exercised on physical hardware in this environment.

Checklist

• I have read and confirmed this PR follows the contribution guidelines.

[leejet]

Thanks for the PR! I kept the core prefetch functionality, but removed a few parts:

• Prefetch depth: I don't think deeper prefetching helps much. If compute is faster than transfer, transfer is still the bottleneck; if compute is slower, a single prefetch is enough to overlap it.
• Resident layers: this should be handled automatically by the algorithm rather than exposed as a user-facing option.
• Eviction/residency and cross-forward state: these are currently too tightly coupled and conflict with the new weight-management model I'm planning.

I also reorganized the code, and the resulting implementation is now much simpler and more straightforward.