Skip to content

WinUI fail-fast 0xc000027b: the GC finalizer thread releases XAML objects (idle crash after ~1h) #513

Open

Description

@iamfatness

The app crashed idle after 58 minutes in a live meeting (Tiles wall on Program, nobody touching it), 2026-09-13 01:11. Full-memory dump: CoreVideoPro.WinUI.exe.19580.dmp (994 MB). PDBs still matched, so it was analyzable.

This is a new member of the 0xc000027b family — a different trigger from all five already documented in CLAUDE.md, and the first one that is asynchronous and time-delayed.

The crash

ExceptionCode: c000027b            (stowed exception fail-fast)
Stowed inner:  8000000e            = E_ILLEGAL_METHOD_CALL
Failing module: Microsoft.UI.Xaml.dll
Crashing thread: 3  — (Finalizer), MTA

Stack, bottom-up — the whole story is in five frames:

coreclr!FinalizerThread::FinalizeAllObjects
System_Private_CoreLib!System.GC.RunFinalizers()
WinRT_Runtime!WinRT.IObjectReference.Finalize()
Microsoft_UI_Xaml!ctl::ComObject<DirectUI::Border>::Release
Microsoft_UI_Xaml!FailFastWithStowedExceptions
combase!RoFailFastWithErrorContextInternal2
KERNELBASE!RaiseFailFastException

The GC finalizer thread released a XAML Border. XAML objects have thread affinity: they belong to the UI thread's STA. The finalizer thread is MTA. Releasing the object's underlying COM pointer from there is an illegal method call, Microsoft.UI.Xaml stows E_ILLEGAL_METHOD_CALL and fail-fasts the process. There is no managed stack in our code, which is why nothing we log could ever have caught it.

Why the finalizer had XAML objects to release

Heap at the moment of death:

finalizable type count
WinRT.ObjectReference<WinRT.Interop.IUnknownVftbl> 2,146
WinRT.ObjectReferenceWithContext<IUnknownVftbl> 268
_EventSource_…RoutedEventHandler+EventState 111
_EventSource_…SizeChangedEventHandler+EventState 23
Microsoft.UI.Xaml.Controls.Border (live) 114

WinRT.ObjectReference<IUnknownVftbl> is exactly the CsWinRT wrapper whose Finalize() is in the stack. 118 objects were "ready for finalization" when it died, with 4,969 more on the gen-1 finalizable queue.

Those wrappers come from XAML elements built in code-behind and then thrown away. Every one of these controls creates elements and discards them with Children.Clear():

control creates hooks += unhooks -=
ShowMultiviewHost (ClickOverlay/DecorOverlay, rebuilt on every structural change) 8 0 0
ScenePreviewControl 17 0 0
AudioLevelMeter (segments rebuilt on refit/resize) 6 0 0
SceneCanvasEditorControl 6 4 0

SceneCanvasEditorControl hooks four events on generated elements and unhooks none.

Why it took 58 minutes, and why no soak has ever caught it

Nothing forces the collection. The wrappers sit on the finalizable queue until a GC actually runs finalizers. The app was idle, so that took ~43 minutes after the last operator action; when the pass finally ran, the queue held 118 ready objects and one of them was a Border.

Every previously documented 0xc000027b trigger was a synchronous UI-thread churn pattern that reproduced while the offending code ran. This one fires long after, on another thread, with no application code on the stack. A 5-minute soak cannot see it; the take/drill/soak gates all passed clean minutes earlier in the same session.

Not caused by the current branch

The session ran 71e4d4dc (the #448 slice), whose changes are entirely C++ core-side. This is shell-side and predates it. The core log's last 8 minutes show only the [lock-guardrail] render-tick line — correlated in time, but a different subsystem and not on the faulting stack.

Fix direction

There is no supported way to marshal a release back to the UI thread from a finalizer, so the fix is to stop discarding XAML elements to the GC:

  1. Reuse instead of recreate. Pool the per-tile decoration elements and diff-update them, exactly the rule this codebase already enforces for bound collections (and for the same crash family). ShowMultiviewHost's overlay rebuild is the highest-traffic offender — it clears and rebuilds on every structural change.
  2. Unhook every handler attached to a generated element before dropping it, starting with SceneCanvasEditorControl's four.
  3. Add a guard test in the shape of the existing Page_GuardsEveryUiCallback tests: fail if a code-behind element factory's output is discarded without teardown.

Repro: launch, exercise the multiview/overlay rebuilds (a handful of takes is enough), then leave the app idle in a meeting for an hour. Expect the fail-fast without touching anything. A forced GC.Collect(); WaitForPendingFinalizers() after a rebuild burst should reproduce it in seconds and makes a regression test possible.

Evidence: CrashDumps\CoreVideoPro.WinUI.exe.19580.dmp; analysis saved in the session scratchpad (dump-analyze.txt, dump-stack.txt, dump-sos.txt, dump-fq.txt).

Activity

  1. iamfatness commented on Sep 13, 2026

    @iamfatness
    OwnerAuthor

    Correction after attempting to reproduce (2026-09-13 06:35–07:10)

    Retracting the repro claim above. "A rebuild burst then GC.Collect(); WaitForPendingFinalizers() should reproduce it in seconds" was a prediction, and it failed twice on the same build in the same meeting:

    experiment forced full GC via dotnet-gcdump collect result
    control: fresh app, no churn yes survived
    12 takes (overlay structural rebuilds) yes survived
    record 20 s → stop → 3 takes yes survived

    Heap deltas show the forced GC did its job: 5,144 → 2,453 WinRT.ObjectReference<IUnknownVftbl> — about 2,700 CsWinRT wrappers finalized on the finalizer thread without incident. So a finalizer-thread release is the ordinary path, not the defect; last night's Border was special.

    What IS confirmed, from the dump (facts, not inference)

    • Native object: Microsoft_UI_Xaml!ctl::ComObject<DirectUI::Border>::vftable (resolved from the wrapper's _thisPtr).
    • Wrapper: a plain WinRT.ObjectReference<IUnknownVftbl> — not ObjectReferenceWithContext — with _referenceTrackerPtr set and PreventReleaseFromTrackerSourceOnDispose = 1. Its release had no context to marshal to.
    • UI thread (thread 0, STA) at that instant: idle in win32u!NtUserGetMessage under FrameworkApplication::RunDesktopWindowMessageLoop. Not blocked, not tearing down — a marshaled release would have landed.
    • Population at death: 66 dead Border RCWs, 1,845 dead plain wrappers, 118 objects ready for finalization. That population is the SAME size as a healthy run (64–66 live Borders here), which finalizes safely.
    • launch.log was silent from 00:28 to the crash at 01:11 — no roster change, no operator action, no window/dialog. The trigger was GC timing alone.
    • Our code has no manual CsWinRT marshaling (ObjectReference.*, MarshalInspectable, As<…Native> — none outside ComWrappersSupport.InitializeComWrappers()), and none of the four element-building controls touches XAML from a Timer, Task.Run, or a ConfigureAwait(false) continuation. Nothing found that would mint a context-less XAML wrapper from our side.
    • Versions: Microsoft.WindowsAppSDK.Runtime 2.4.0 / .WinUI 2.3.6, WinRT.Runtime.dll 2.2.0.

    Revised assessment

    The crash is a framework-side release-ordering fault in the CsWinRT reference-tracker path (WinAppSDK 2.x + CsWinRT 2.2): a XAML Border whose tracker-backed wrapper was finalized while the XAML peer was in a state that refuses an off-thread final release. What put that one Border in that state is not established and did not reproduce under forced GC. The four Children.Clear() element factories remain the likely population, not a proven cause; the "fix direction" above (pool/diff-update, unhook handlers) reduces exposure but I cannot claim it eliminates the crash.

    Natural repro in progress: the same build is running idle in the test meeting since 06:36 (pid 42148) with the same churn history. If it dies the same way with nothing touching it, that is the reproduction; the dump will be analyzable as long as nobody rebuilds the shell.

  2. added 2 commits that reference this issue on Sep 13, 2026
  3. iamfatness commented on Sep 17, 2026

    @iamfatness
    OwnerAuthor

    Live verification 2026-09-17, installed beta-2026-09-15-e15413c; Windows RTX 4090 / i7-14700K. Eight camera guests, 1080p60 configured YouTube RTMP, simultaneous Program + eight ISO recordings, per-guest audio mode, four Takes. Recording duration 221.55 seconds, measured interval 178.295 seconds.

    No app crash observed in this finite live test. App had already been running approximately 15 hours before the test and remained open afterward. This does not close the idle XAML crash or establish long-event reliability.

  4. iamfatness commented on Sep 18, 2026

    @iamfatness
    OwnerAuthor

    PR #548 (exposure reduction, not closure): SceneCanvasEditorControl now unhooks the four per-frame pointer handlers before removing a stale layer frame, and OnPresetClick before clearing preset buttons — fix-direction #2 — plus a source-analysis guard test (fix-direction #3) that fails if a per-element handler is added without teardown. WinUI builds clean, 3 guard tests pass. This is the editor-path leak; the idle-crash population is dominated by ShowMultiviewHost overlay discards during a live show (pooling that is a bigger change to validate live), and the root fault is the CsWinRT/WinAppSDK finalizer release-ordering bug. Recommend keeping this open; the long idle soak remains the gate.

  5. added a commit that references this issue on Sep 18, 2026
  6. iamfatness commented on Sep 18, 2026

    @iamfatness
    OwnerAuthor

    PR #549: ShowMultiviewHost overlays (click buttons + per-tile tally/label/meter decorations) are now pooled — created once, updated in place, Visibility-toggled — instead of cleared and recreated on every structural change. This is the dominant discarded-XAML population during a live show (roster/active-speaker/tally churn). Element construction is byte-identical to the former builders, so the rendered multiview is unchanged; only the lifecycle differs. Builds clean, 55 WinUI tests pass, guard test locks it in. Draft-quality gate: needs a live soak before merge since it edits the on-air multiview — please confirm tally/labels/meters track correctly under churn. Exposure reduction, not closure; keep this open. With #548 (editor path) merged, this removes the two largest discarded-element sources.

  7. added a commit that references this issue on Sep 18, 2026
  8. iamfatness commented on Sep 19, 2026

    @iamfatness
    OwnerAuthor

    Idle soak of beta-2026-09-19-e07f9f5 (includes #548 editor teardown + #549 multiview pooling), 2026-09-19 15:00–17:31 EDT.

    Setup per the repro in this issue: packaged app run from its release folder, joined the test meeting (8 guests with video), Engine on, 7 Tiles↔speaker-slides takes to churn the multiview/overlay rebuilds, then left the Tiles wall on Program and did not touch it.

    Result: alive at 151 minutes idle (watchdog cap), no fail-fast, no new CoreVideoPro.WinUI.exe dump in %LOCALAPPDATA%\CrashDumps (full-dump WER key confirmed armed). Previous occurrence fired at 58 min.

    Memory over the run (working set, 5-min samples): shell 456 → 517 MB, core 351 → 463 MB — a slow drift in the first 30 min then a plateau ~500/465 MB.

    This is one clean run, not closure (the crash is a GC-finalizer timing event; see the "stability claims are bounded by the watched window" rule). It does say the pooling/teardown work moved the exposure enough that the 58-minute reproduction did not recur under the same recipe. Suggest two more idle runs across different sessions before downgrading this from Now #1.

  9. iamfatness commented on Sep 21, 2026

    @iamfatness
    OwnerAuthor

    Owner update 2026-09-21: the app has run overnight without another idle crash. Defer further #513 validation until the owner resumes it; work #535 and #555 now. This is useful operator evidence, not root-cause closure. No exact overnight duration or instrumented counter capture was supplied.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backlogRanked in docs/BACKLOG.md

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions