Invoke debugger function evaluations through managed UCO

[jkoritzinsky]

Summary

Layer 3 of the stack for #123864, following #133658. The prior func-eval-specific proposal is #126809.

This layer started as replacing the final native managed-call dispatch with a managed UnmanagedCallersOnly entrypoint. It now moves most debugger function-evaluation preparation, invocation, storage, and copy-back into managed CoreLib while retaining the debugger-specific control flow and GC-safe bootstrap in native code.

Managed invocation and storage

• Add Debugger.FunctionEvaluation, a runtime-owned UCO entrypoint that resolves the exact target method from its declaring RuntimeType and rooted IRuntimeMethodInfo.
• Extend the shared reflection invocation emitter with debugger-specific thunks that prepare arguments, invoke methods and constructors, copy back writable arguments, and publish results.
• Emit newobj for debugger object construction instead of preallocating an object and calling its constructor as an ordinary method. Existing-object constructor calls remain supported, and abstract/interface allocation is rejected.
• Keep the entire managed call-address vector registered through GCFrameRegistration, using exact typed locals for memory-backed by-value ref structs so their managed byrefs are reported by normal JIT GC info.
• Handle object, primitive, enum, pointer, value-type, nullable, literal, memory, register, and byref argument representations in managed code while preserving native-endian storage and debugger copy-back semantics.
• Align nullable behavior with the existing debugger contract: direct value-type memory homes are mutated in place, while debugger handles are not treated as writable nullable homes. Exact register-backed nullable temporaries are boxed only after their MethodTable is instance-active.
• Move ordinary value-type result storage into temporary GC-described external memory and defer boxing until after invocation and argument copy-back. Keep byref-like results in persistent external storage and raw byref results rooted directly in the debugger result slot.
• Create result handles in managed code only after the result is fully prepared, and preserve object identity and updated field contents across collections and later debugger inspection.

Native debugger and GC integration

• Retain native method/signature validation, generic resolution, class initialization, receiver validation inputs, register/home capture, abort state, exception publication, completion, and result ownership.
• Capture contiguous register value-type snapshots before the first GC-triggering managed transition and describe them with their exact type layout.
• Root non-leaf register references and register value-type snapshots with ExternalMemoryHandle registrations so collections during managed preparation relocate their contents correctly.
• Represent external memory with a full tagged TypeHandle, allowing the runtime and cDAC to distinguish inline value types, object-reference slots, managed-byref slots, and unmanaged pointer/function-pointer locals.
• Report each external-memory type's loader allocator during GC scanning, keeping collectible layouts alive while external storage is registered.
• Add dedicated GC-safe object-register copy-back and native-endian scalar register copy-back paths.
• Remove stale native argument copies and architecture-specific packing layers that are no longer needed, while preserving the architecture-specific register-home logic required by debugger IPC.
• Treat the debugger UCO entrypoint as a runtime-internal reverse transition for exception handling, and exclude its debugger-only MethodDesc state on WebAssembly targets that do not build debug/ee.
• Update the cDAC ExternalMemoryHandles reader, descriptors, tests, and authoritative contract documentation for the exact managed-local representation.

Correctness fixes included

• Avoid double relocation of overlapping byrefs in by-value ref structs (Debugger func-eval: a by-value ref struct with overlapping ref fields is reported to the GC twice, so a compacting GC relocates the byref twice #135245).
• Keep raw byref results rooted until deferred debugger completion (Debugger func-eval byref result can become stale before deferred completion #135247).
• Prevent stale non-leaf register values after a compacting GC (Debugger func-eval can use stale non-leaf register values after GC #135249).
• Preserve current external-result fields across collections and later inspection (Debugger external-memory func-eval results can expose stale cached fields after GC #135251).
• Keep collectible loader allocators alive while external-memory layouts are scanned (Func-eval ref struct result: ExternalMemoryHandle doesn't keep a collectible LoaderAllocator alive (access violation in GC after unload) #135252).

Validation

Validation used the existing internal MDbg/xUnit debugger-test harness with a private CoreRun and ProjectK/CoreCLR-only configuration:

• Targeted nested-byref invariant under target-only GCStress=0x3: 1/1 passed.
• Revised normal matrix: 35/35 passed.
• Targeted stress matrix: 4/4 passed.
• Windows x64 and x86 Checked native, CoreLib, and ReadyToRun builds completed with zero warnings and errors.
• cDAC unit tests and generated usage/documentation checks passed for the ExternalMemoryHandle descriptor changes.

Coverage includes moving GC, overlapping and nested byrefs, memory and register homes, non-leaf registers, receiver mutation, constructors, primitive/native-endian copy-back, nullable memory homes, byref/byref-like/value-type results, collectible generics and layouts, nested and exception-time eval, safe/rude/exception-unwind aborts, and post-eval inspection. Runtime hashes and loaded modules were audited; each abort scenario completed the callback, successfully evaluated Echo(42), detached, and exited MDbg with code 100. Test configurations were restored and no debugger/test processes survived the runs.

These results were recorded on the validated pre-refresh candidate. The final stack rebase preserved the layer's net diff; builds and tests were not repeated after the history rewrite at the user's request.

The final native invocation cleanup layer is #133660.

---

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

Resolves #135245
Resolves #135247
Resolves #135249
Resolves #135251
Resolves #135252

Note

This PR description and implementation were prepared with GitHub Copilot. Commits that reuse am11's code retain the requested co-author attribution.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

[am11]

discussed on #126542.

nit: funceval was already split: #126809 so #126542 discussion is irrelevant.

[jkoritzinsky]

Thanks, @am11. Corrected the summary to reference #126809 for the prior func-eval-specific proposal rather than #126542.

Note

This reply was prepared with GitHub Copilot.

[jkotas]

This fix should be submitted as a separate PR with a targeted regression test. It is not a debugger specific issue. For example, the following is expected to print 1 2 3 4, but it prints bogus numbers on win x86 with DOTNET_GCStress=1 .

using System;
using System.Runtime.InteropServices;

X x = default;
x.a.a = new int[10]; x.a.a.Fill(1);
x.a.b = new int[10]; x.a.b.Fill(2);
x.a.c = new int[10]; x.a.c.Fill(3);
x.a.d = new int[10]; x.a.d.Fill(4);
Test(x);

static void Test(X x)
{
    Console.WriteLine(x.a.a[0]);
    Console.WriteLine(x.a.b[0]);
    Console.WriteLine(x.a.c[0]);
    Console.WriteLine(x.a.d[0]);
}

ref struct A
{
    public Span<int> a,b,c,d;
}

ref struct B
{
    public Span<int> a,b,c,d;
}

[StructLayout(LayoutKind.Explicit)]
ref struct X
{
    [FieldOffset(0)]
    public A a;

    [FieldOffset(0)]
    public A b;
}

[jkotas]

This is quadratic algorithm that can take a while if somebody created large ExplicitLayout struct. It is probably ok given that this should be a very rare situation and large structs have sharp edges in general, just pointing it out. It may be worth a comment

[jkotas]

Suggested change

	#if defined(DEBUGGING_SUPPORTED) && !defined(TARGET_WASM)
	#if defined(DEBUGGING_SUPPORTED)