Add additional round-trip fixes and add additional unit tests for greater code coverage - #133188
Open
jkoritzinsky wants to merge 20 commits into
Open
Add additional round-trip fixes and add additional unit tests for greater code coverage#133188jkoritzinsky wants to merge 20 commits into
jkoritzinsky wants to merge 20 commits into
Conversation
Use the root command option schema for value arity and aliases, and pass ParseResult directly into Program so the schema can be shared with tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 69313a92-28b5-4e32-8be8-ea4f6e44a5c8
Use content-based portable PDB IDs for deterministic builds and add in-process coverage for byte-identical PDB output and non-empty PDB ID GUIDs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2fb9f1ef-f721-49e0-bf48-4a7a9458d53a
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2fb9f1ef-f721-49e0-bf48-4a7a9458d53a
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2fb9f1ef-f721-49e0-bf48-4a7a9458d53a
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2fb9f1ef-f721-49e0-bf48-4a7a9458d53a
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2fb9f1ef-f721-49e0-bf48-4a7a9458d53a
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 106f96fe-da32-41d4-8aa5-7431edb36fc6
…ompiler, it's a compiler that's AOT compiled)
Report oversized generic parameter declarations and constraint owners while preserving native ilasm-compatible wrapped parameter rows in error-tolerant output. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 286b5b04-7d1b-4228-ad6b-dac025ce91a7
Add feature-focused metadata and PE validation tests and fix emitter issues exposed by them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4307156b-37a7-495d-9af6-fe9174c05b0b
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
… attribute encoding When an object(...) element in an object[] custom attribute had nested object() wrappers (e.g. object(object(bool(true)))), VisitSerInit's object() branch would call WriteCustomAttributeFieldOrPropType on the inner object() context, producing TaggedObject (0x51) as the element type code. The .NET metadata decoder does not accept 0x51 as a FieldOrPropType for object[] elements and throws BadImageFormatException. Fix VisitObjSeq to fully unwrap any object(...) nesting using a while loop, so that WriteCustomAttributeFieldOrPropType is always called with the innermost concrete typed element (bool, int32, string, etc.). This also makes type-byte handling uniform - always done by VisitObjSeq rather than split between caller and callee. Add a regression test CustomAttribute_ObjectArrayWithNestedObjectWrapper _DecodesProperly that exercises object[](object(bool(true)) object(int32 (42))) and verifies the attribute decodes correctly.
Use comma separators between double-quoted GUID arguments so the tests match the supported .language grammar. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: adc580d5-8f1c-4772-bfc2-7a943aa0959d
Pre-size floating-point sequence builders and transfer their arrays without copying. Handle empty floating-point and string sequences and cover their custom attribute encoding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: adc580d5-8f1c-4772-bfc2-7a943aa0959d
Allow tests to opt out of managed ilasm round trips while preserving native ilasm coverage, and make the existing all-round-trip exclusion imply the managed-only property. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 286b5b04-7d1b-4228-ad6b-dac025ce91a7
…ely product changes. Make the necessary fixes to the GrammarVisitor model to get these tests passing
|
Azure Pipelines: Successfully started running 5 pipeline(s). 11 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
|
Tagging subscribers to this area: @JulieLeeMSFT, @dotnet/jit-contrib |
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The CLI normalization refactor currently misclassifies value-taking options (using Option.Name vs aliases), and Options introduces a breaking public member rename that should be made compatible or internal.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (3)
| Severity | Finding |
|---|---|
 |
src/tools/ilasm/src/ilasm/NativeCommandLine.cs — optionsWithValues is populated with Option.Name (e.g. "output"), but later checked against… |
 |
src/tools/ilasm/src/ILAssembler/Options.cs — Options is a public type and this change replaces the existing public IsDll property with Dll.… |
 |
src/tools/ilasm/tests/ILAssembler.Tests/CommandLineTests.cs — ModernValueOptions currently includes option.Name in addition to prefixed aliases. In… |
What changed in this PR
This PR significantly expands the managed ilasm test suite and adjusts the managed IL assembler, test infrastructure, and CI/configuration to support additional IL round-trip scenarios (including a new managed-ilasm roundtrip), while fixing multiple metadata/PE emission and parsing edge cases uncovered by the added coverage.
Changes:
- Add broad ILAssembler unit test coverage across metadata emission (typedefs/typerefs/exports/resources/events/properties/fields), parser robustness, preprocessor integration, and options/CLI behavior.
- Update managed
ilasmimplementation to improve correctness/compatibility for opcode parsing, vararg/signature handling, PrivateScope naming, PE export/vtable fixup emission, and error-tolerant output behavior. - Extend CoreCLR test infrastructure/CI to support
managedilasmroundtripand introduce opt-out markers (ManagedIlasmRoundTripIncompatible) with accompanying documentation.
| File | Description |
|---|---|
| src/tools/ilasm/tests/ILAssembler.Tests/TypeSignatureTests.cs | Adds tests for multi-dimensional array local signature shape decoding. |
| src/tools/ilasm/tests/ILAssembler.Tests/TypeReferenceTests.cs | Adds tests for typeref resolution scopes, mdtoken/.this/.base/.nester, module-scoped typerefs, and typelist parsing. |
| src/tools/ilasm/tests/ILAssembler.Tests/TypedefTests.cs | Adds typedef alias coverage for fields and custom attributes (including owner-scoped forms). |
| src/tools/ilasm/tests/ILAssembler.Tests/TypeDefinitionTests.cs | Adds tests for quoted name segments, class attribute flag behavior, nested types, and interface impl emission. |
| src/tools/ilasm/tests/ILAssembler.Tests/SyntaxTests.cs | Adds corpus-style tests ensuring malformed/truncated inputs report diagnostics instead of throwing. |
| src/tools/ilasm/tests/ILAssembler.Tests/SourceDirectiveTests.cs | Strengthens verification of escape processing and adds portable PDB directive coverage across variants/multi-doc scenarios. |
| src/tools/ilasm/tests/ILAssembler.Tests/SecurityTests.cs | Adds coverage for permissionset syntax variants, emitted DeclSecurity metadata, and error-tolerant behavior. |
| src/tools/ilasm/tests/ILAssembler.Tests/PropertyTests.cs | Adds tests for property attributes/accessors/custom attributes and indexed property signatures. |
| src/tools/ilasm/tests/ILAssembler.Tests/PropertyEventTests.cs | New tests validating property/event maps and MethodSemantics emission. |
| src/tools/ilasm/tests/ILAssembler.Tests/PreprocessorIntegrationTests.cs | New integration tests for includes, conditional compilation, and macro expansion across include levels. |
| src/tools/ilasm/tests/ILAssembler.Tests/ParameterTests.cs | Adds tests for parameter attribute syntaxes emitting expected flags. |
| src/tools/ilasm/tests/ILAssembler.Tests/NativeExportTests.cs | New tests validating PE export directory/stub emission and machine-specific stub prefixes. |
| src/tools/ilasm/tests/ILAssembler.Tests/ModuleTests.cs | Adds tests for .file metadata/resources, module refs, bare .module, and global method/module attribute + PDB emission. |
| src/tools/ilasm/tests/ILAssembler.Tests/ManifestResourceTests.cs | New tests for embedded/file/assembly-backed manifest resources and deterministic/error-tolerant behavior. |
| src/tools/ilasm/tests/ILAssembler.Tests/ILAssembler.Tests.csproj | Adds System.CommandLine dependency and links IlasmRootCommand.cs for CLI parsing tests. |
| src/tools/ilasm/tests/ILAssembler.Tests/FunctionPointerTests.cs | Switches to signature decoding assertions; adds vararg fnptr sentinel coverage. |
| src/tools/ilasm/tests/ILAssembler.Tests/FieldTests.cs | Adds tests for trailing custom attribute binding and field attribute flag/null constant emission. |
| src/tools/ilasm/tests/ILAssembler.Tests/ExportedTypeTests.cs | Adds coverage for ExportedType attributes/implementations/custom attributes and TypeDefId emission. |
| src/tools/ilasm/tests/ILAssembler.Tests/ExceptionHandlingTests.cs | Adds extensive EH region tests (offset-based, label-based, ordering, filter/fault/finally/catch). |
| src/tools/ilasm/tests/ILAssembler.Tests/EventTests.cs | Adds event accessor/attribute/other/custom-attr metadata tests and nil-type behavior. |
| src/tools/ilasm/tests/ILAssembler.Tests/DocumentCompilerTests.cs | New tests ensuring diagnostics have correct document paths and parser errors don’t leak scopes across documents. |
| src/tools/ilasm/tests/ILAssembler.Tests/DocumentCompilerTestHelpers.cs | Adds shared PEReader compile helper, token operand extraction, and metadata signature decoding provider. |
| src/tools/ilasm/tests/ILAssembler.Tests/DataTests.cs | Adds coverage for data directives (typed bytes, repeats, large bytearrays, float literals, syntax variants, error isolation). |
| src/tools/ilasm/tests/ILAssembler.Tests/CompilerOptionsTests.cs | New tests verifying Options-driven PE/metadata behavior, debug attributes, deterministic output, keyfile handling, and error-tolerant output. |
| src/tools/ilasm/tests/ILAssembler.Tests/CommandLineTests.cs | Updates modern option corpus generation to reflect IlasmRootCommand options (CLI normalization tests). |
| src/tools/ilasm/tests/ILAssembler.Tests/AssemblyTests.cs | Updates Options property name usage and adds additional assembly reference/identity/attribute parsing tests. |
| src/tools/ilasm/src/ILAssembler/VTableExportPEBuilder.cs | Fixes COR header vtable directory offset and improves vtable/export directory emission logic (including ordinal gaps). |
| src/tools/ilasm/src/ILAssembler/PreprocessedTokenSource.cs | Fixes include-stack EOF handling to avoid incorrectly popping the entry source. |
| src/tools/ilasm/src/ILAssembler/Options.cs | Renames the DLL output option property (public surface) and updates option model. |
| src/tools/ilasm/src/ILAssembler/NameHelpers.cs | Adds helper to normalize PrivateScope metadata names (strip $PST + token suffix). |
| src/tools/ilasm/src/ILAssembler/MetadataExtensions.cs | Adds ILOpCode.Unused support used by opcode parsing. |
| src/tools/ilasm/src/ILAssembler/gen/CIL.g4 | Fixes token spelling and extends grammar for dotted names, mdtoken forms, unsigned native types, permissionset syntax, and opcode set. |
| src/tools/ilasm/src/ILAssembler/EntityRegistry.cs | Updates metadata emission behavior (PrivateScope naming, method body encoding heuristics, methodimpl owner, vararg sig rewriting, error-tolerant metadata row preservation). |
| src/tools/ilasm/src/ILAssembler/DocumentCompiler.cs | Enhances parser error context propagation so the visitor can avoid leaking state across erroneous contexts. |
| src/tools/ilasm/src/ILAssembler/Diagnostic.cs | Adds a new diagnostic ID/message for constraint owner out-of-range scenarios. |
| src/tools/ilasm/src/ilasm/Program.cs | Refactors CLI handling to use ParseResult explicitly and wires Normalize() with a single command instance. |
| src/tools/ilasm/src/ilasm/NativeCommandLine.cs | Refactors CLI normalization to derive supported options from IlasmRootCommand. |
| src/tools/ilasm/src/ilasm/IlasmRootCommand.cs | Removes embedded SetAction wiring (now configured from Program.Main). |
| src/tools/ilasm/src/ilasm/ilasm.csproj | Inlines NativeAOT runtime settings previously imported from AotCompilerCommon.props. |
| src/tests/Loader/classloader/regressions/dev10_403582/gentype.ilproj | Marks the project as incompatible with managed ilasm round-trip. |
| src/tests/Loader/classloader/regressions/dev10_403582/genmeth.ilproj | Marks the project incompatible with IL round-trip (and documents ildasm crash). |
| src/tests/Directory.Build.targets | Emits marker files for IlasmRoundTripIncompatible and ManagedIlasmRoundTripIncompatible. |
| src/tests/Common/CLRTest.Jit.targets | Adds marker-file based skipping and introduces managed-ilasm round-trip selection logic. |
| src/tests/Common/CLRTest.Execute.targets | Makes IlasmRoundTripIncompatible imply ManagedIlasmRoundTripIncompatible. |
| src/tests/Common/CLRTest.Execute.Batch.targets | Adds managed-ilasm-only skip logic for batch execution wrapper. |
| src/tests/Common/CLRTest.Execute.Bash.targets | Adds managed-ilasm-only skip logic for bash execution wrapper. |
| eng/pipelines/common/templates/runtimes/run-test-job.yml | Adds managedilasmroundtrip scenario to ilasm test group. |
| docs/workflow/testing/coreclr/test-configuration.md | Documents ManagedIlasmRoundTripIncompatible and the implication behavior. |
| docs/workflow/testing/coreclr/requiresprocessisolation.md | Updates guidance for IL round-trip incompatibility properties and isolation requirements. |
| docs/workflow/ci/disabling-tests.md | Documents disabling managed vs all ilasm round-trip and notes merged-runner behavior. |
Comment on lines
+60
to
+67
| foreach (Option option in command.Options) | ||
| { | ||
| if (option.Arity.MinimumNumberOfValues > 0) | ||
| { | ||
| optionsWithValues.Add(option.Name); | ||
| modernOptionsWithSeparateValues.Add(option.Name); | ||
| modernOptionsWithSeparateValues.UnionWith(option.Aliases); | ||
| } |
Comment on lines
+68
to
+72
| /// <summary> | ||
| /// Produce a DLL image instead of an executable. | ||
| /// </summary> | ||
| public bool Dll { get; set; } | ||
|
|
Comment on lines
+68
to
+80
| foreach (Option option in new IlasmRootCommand().Options) | ||
| { | ||
| if (option.Arity.MinimumNumberOfValues == 0) | ||
| { | ||
| continue; | ||
| } | ||
|
|
||
| options.Add(option.Name); | ||
| foreach (string alias in option.Aliases) | ||
| { | ||
| options.Add(alias); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Creating a new PR to replace #131509.