Skip to content

Route eager op outputs through ctx.memoryScope (DefaultCpuOps + decode-loop adoption) #1146

Closed

Description

@michalharakal

Follow-up of #1135 (adoption engineering; the design question is settled there). With ExecutionContext.memoryScope wired into tensor creation, eager op outputs still bypass it: DefaultCpuOps.kt has ~34 raw FloatArray(...) allocation sites wrapped via Storage.Heap.wrap (Owner.Borrowed, never freed), and no ctx threading reaches most of them.

Scope

  • Thread the active scope into op output allocation (DefaultCpuOps and friends) so a ForwardScope actually captures activations, not just user-created tensors.
  • Generation-loop adoption in a real model path (SKaiNET-transformers side or DecodeHarness), with reset() per step.
  • Preserve/validate the raw-buffer fast paths (offset-0 assumption) and measure: peak allocation per decode step before/after; slab sizing from peakFloats.

Acceptance

  • DecodeHarness-level test showing steady-state decode allocates ~zero new heap per step under a ForwardScope.
  • No numeric drift vs Ambient; no perf regression on the JVM leg.

Metadata

Metadata

Assignees

No one assigned

    Labels

    codingImplementation task (DARC: C)tensorsTensor operations and data structures

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions