Skip to content

Fix double GC relocation of overlapping byrefs - #135890

Merged
jkoritzinsky merged 3 commits into
dotnet:mainfrom
jkoritzinsky:dev/jkoritzinsky/overlapping-byref-gc-fix
Oct 17, 2026
Merged

jkoritzinsky merged 3 commits into
dotnet:mainfrom
jkoritzinsky:dev/jkoritzinsky/overlapping-byref-gc-fix

Conversation

@jkoritzinsky

Copy link
Copy Markdown
Member

Summary

Report each physical byref slot only once when scanning an explicit-layout byref-like value type. Overlapping fields currently cause the shared scanner to report the same slot multiple times, so a compacting GC can relocate an already-relocated interior pointer again.

This is the standalone GC fix requested in review of #133659. It is not debugger-specific: an ordinary by-value ref-struct argument reproduces the corruption on Windows x86 with DOTNET_GCStress=1.

  • Track enclosing overlapping layouts and check earlier fields for the exact slot address, not the referenced object value. The check is stack-only and allocation-free; document its quadratic cost for rare overlapping layouts.
  • Use the already-loaded value-type field handle rather than triggering type loading during GC enumeration.
  • Add a targeted test to the existing ref-field tests, asserting all four expected values through both overlapping views. A non-inlined callee preserves the by-value argument across the JIT transition.

No debugger-specific or cDAC changes are included.

Validation

Windows x86 Checked runtime, Release libraries, at main base 6b1cd4b85c4b:

Check Unfixed main Fixed
Reviewer's repro, normal 1 2 3 4 1 2 3 4
Reviewer's repro, DOTNET_GCStress=1 0 3801155 7536732 2949173 1 2 3 4
Ref-field tests, normal 6/6 passed 6/6 passed
Ref-field tests, DOTNET_GCStress=1 New regression fails: expected 1, actual 6881312 (exit 101) 6/6 passed (exit 100)
Existing inline-array tests, normal / DOTNET_GCStress=1 Not run 11/11 passed in each mode (exit 100)

The regression assembly and CoreLib are byte-identical between the unfixed and fixed runs.

Build commands and results:

.\build.cmd clr+libs+host -a x86 -rc Checked -lc Release
# Passed; 0 errors, 3 existing MsQuic NU1902 warnings.
.\build.cmd clr.native+clr.corelib+clr.nativecorelib -c Checked -a x86
# Passed; 0 warnings, 0 errors.
.\src\tests\build.cmd -GenerateLayoutOnly x86 Checked /p:HostConfiguration=Debug
# Passed before and after the fix.
.\src\tests\build.cmd -Test Loader\classloader\RefFields\Validate.csproj x86 Checked /p:HostConfiguration=Debug
# Passed; 1 selected test project, 0 warnings/errors.
.\dotnet.cmd build -c Checked src\tests\Loader\classloader\InlineArray\InlineArrayValid.csproj /p:TargetArchitecture=x86 /p:BuildAsStandalone=true
# Passed; 0 warnings/errors.

Tests were run with CORE_ROOT set to artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root, using & "$env:CORE_ROOT\corerun.exe" with Validate.dll and InlineArrayValid.dll from their respective output directories, once without DOTNET_GCStress and once with it set to 1. Other architectures/platforms were not run.

Resolves #135245

Note

This implementation and PR description were prepared with GitHub Copilot.

Deduplicate physical byref slots in explicit-layout value types without allocating during GC. Preserve non-loading field lookups and document the rare-layout quadratic scan. Add a targeted by-value regression in the existing ref-field tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

Nest ValueClassByRefSearchContext inside ByRefPointerOffsetsReporter without changing its scanning behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Comment thread src/tests/Loader/classloader/RefFields/Validate.cs
Comment thread src/coreclr/vm/siginfo.cpp Outdated
Reuse the value-type reporter in overlap-probe mode without building nested overlap lists or recursively checking overlaps. Add nested-layout regression coverage with distinct and shared backing arrays.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Safety-critical GC scanning changes need runtime-owner review of scanner invariants and platform coverage beyond Windows x86.

No open issues 🎉

What changed in this PR

Fixes double GC relocation by reporting each physical byref slot once in overlapping ref-struct layouts, independently of debugger functionality.

Changes:

  • Deduplicates overlapping slots using allocation-free traversal.
  • Uses already-loaded field type handles during GC scanning.
  • Adds regression coverage for nested overlaps and shared backing arrays.

Validation reported in the PR covers Windows x86; no builds or tests were run during this review.

File Description
src/​tests/​Loader/​classloader/​RefFields/​Validate.cs Tests overlapping and nested byrefs through by-value arguments.
src/​coreclr/​vm/​siginfo.cpp Prevents duplicate slot reporting and avoids type loading during scanning.

Review effort: Balanced

@jkoritzinsky

Copy link
Copy Markdown
Member Author

/ba-g failures unrelated

@jkoritzinsky
jkoritzinsky merged commit 6fe708a into dotnet:main Oct 17, 2026
106 of 108 checks passed
@github-project-automation github-project-automation Bot moved this to Done in AppModel Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Debugger func-eval: a by-value ref struct with overlapping ref fields is reported to the GC twice, so a compacting GC relocates the byref twice

3 participants