Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions docs/design/mono/webcil.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,29 @@ the relocation scheme in its own code, reducing the volume of code needed in eac
images into linear memory, as well as for allowing for efficient storage of 128 bit vector constants
within the binary.)

##### WASI host composition

WASI hosts use offline composition rather than instantiating R2R modules at runtime.
The in-tree C# composer unbundles the host component, creates a shim exporting the image and
table bases, merges the host, shim, and composite with `wasm-merge`, folds the globals with
`wasm-opt --simplify-globals`, and replaces the component's core module. Both Binaryen steps
preserve the `name` section with `-g`.

The host exports its reserved buffer address, buffer capacity, and composite table base through
`wasi_r2r_image_base`, `wasi_r2r_image_cap`, and `wasi_r2r_table_base`. Before merging, the composer
checks that the active payload fits the buffer and that the composite's table entries end before
the host's own active element segment. These checks must happen before instantiation: a runtime
check cannot prevent an active segment from overwriting an undersized reservation.

The shim's start function calls `patchWebcilHeader`; the active segments themselves install the
payload and function table. The host's external assembly probe serves `composite-r2r.wasm` from
the embedded buffer and maps raw per-assembly WebCIL forwarding stubs from `comp/<assembly>.dll`.
The composer extracts those payloads from their self-installing Wasm wrappers at build time.
Placing a composite on disk without composing it into the host does not satisfy this contract.

Publishing sizes the host reservations from the generated app/framework composite.
See [the WASI R2R workflow](../../workflow/building/coreclr/wasi-r2r.md) for usage and diagnostics.

### Webcil payload

The webcil payload contains the ECMA-335 metadata, IL and resources comprising a .NET assembly.
Expand Down
1 change: 1 addition & 0 deletions docs/workflow/building/coreclr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ Detailed instructions on how to do cross-compilation can be found in the cross-b
For specialized platforms, detailed instructions are available in the following guides:

- **WebAssembly:** [Building CoreCLR for WebAssembly](/docs/workflow/building/coreclr/wasm.md) - Experimental support for building, running, and debugging CoreCLR on WebAssembly.
- **WASI ReadyToRun:** [Publishing composite R2R images](wasi-r2r.md) - Experimental CoreCLR-WASI composition workflow.

## Other Features

Expand Down
47 changes: 47 additions & 0 deletions docs/workflow/building/coreclr/wasi-r2r.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# CoreCLR-WASI composite ReadyToRun

This is an experimental in-tree publishing workflow. The shipping WASI SDK
does not yet select the CoreCLR app builder.

WASI requires the compiled composite to be composed into the host component before execution.
Copying `composite-r2r.wasm` beside an unmodified host is not sufficient. Composition is
implemented by the in-tree `ComposeWasiReadyToRun` MSBuild task.
The [WebCIL design document](../../../design/mono/webcil.md#wasi-host-composition) describes
the image layout and host contract.

## Prerequisites

Build prerequisites are described in the [CoreCLR build guide](README.md).
Composition additionally requires `wasm-tools` and Binaryen's `wasm-merge` and `wasm-opt`.
Running the result requires wasmtime with WebAssembly exception support.
In-tree publishing acquires pinned tool versions into the shared wasm tool cache through
`eng/AcquireWasiR2RTools.targets`.

Framework-sized composites can require several GiB of memory during composition. Allow for the
host and composite working sets when sizing build containers.

## Publishing

Build the runtime, libraries, and packs, then publish an in-tree WASI project:

```bash
./build.sh -s clr+libs+packs -os wasi -arch wasm -c Release
./dotnet.sh publish <project> -c Release -p:TargetOS=wasi \
-p:RuntimeFlavor=CoreCLR -p:PublishReadyToRun=true
```

The app builder enables composite R2R, compiles the app/framework closure, sizes the host's image
buffer and table reservation, links the host, and invokes the composer. It deploys the composed
host and the per-assembly stubs under the app bundle's `managed/` directory.
Non-composite R2R and `WasmSingleFileBundle` are not supported by this path.

## Composer diagnostics

The build targets generate Crossgen2 response files; a hand-maintained response-file template
is not required. The host supplies the image and table bases, and the C# composer rejects an
undersized buffer, overlapping table reservation, or a non-self-installing composite.

A valid composed module and a successful run alone do not prove R2R was used. Enable the guest's
`DOTNET_ReadyToRunLogFile` and look for `Ready to Run initialized successfully` for the app
assemblies. This confirms image loading; proving a particular method executes compiled code
requires a breakpoint or trace in that method's wasm body.
66 changes: 66 additions & 0 deletions eng/AcquireWasiR2RTools.targets
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
<!--
Acquires the host tools used to compose a CoreCLR-WASI composite ReadyToRun image into the app
host (wasm-tools, and Binaryen's wasm-merge and wasm-opt) into the shared cache described in
eng/wasm/WasmToolCache.props.
-->
<Project>
<PropertyGroup>
<_WasiR2RToolOS Condition="'$(HostOS)' == 'linux'">linux</_WasiR2RToolOS>
<_WasiR2RToolOS Condition="'$(HostOS)' == 'osx'">macos</_WasiR2RToolOS>
<_WasiR2RToolArch Condition="'$(BuildArchitecture)' == 'x64'">x86_64</_WasiR2RToolArch>
<_WasiR2RToolArch Condition="'$(BuildArchitecture)' == 'arm64'">aarch64</_WasiR2RToolArch>
<_BinaryenArch>$(_WasiR2RToolArch)</_BinaryenArch>
<_BinaryenArch Condition="'$(HostOS)' == 'osx' and '$(BuildArchitecture)' == 'arm64'">arm64</_BinaryenArch>
<_BinaryenArchiveName>binaryen-version_$(BinaryenVersion)-$(_BinaryenArch)-$(_WasiR2RToolOS)</_BinaryenArchiveName>
<_WasmToolsArchiveName>wasm-tools-$(WasmToolsVersion)-$(_WasiR2RToolArch)-$(_WasiR2RToolOS)</_WasmToolsArchiveName>

<_BinaryenSha256 Condition="'$(WasmToolHostRid)' == 'linux-x64'">0a18362361ad05465118cd8eeb72edaeec89de6894bc283576ef4e07aa3babcc</_BinaryenSha256>
<_BinaryenSha256 Condition="'$(WasmToolHostRid)' == 'linux-arm64'">e6ae6e09ac40f4e14bc5be6f687c58e2995c84170013975fa641809dd3b480a0</_BinaryenSha256>
<_BinaryenSha256 Condition="'$(WasmToolHostRid)' == 'osx-x64'">d3e2d1235b70c93c54b52eabc1625ea960965152218754f1f4eeb0f873c48e03</_BinaryenSha256>
<_BinaryenSha256 Condition="'$(WasmToolHostRid)' == 'osx-arm64'">79d3ab9f417d9e215f15f598f523d001a7d9ac1e59367e5c869fbdabd1cba72e</_BinaryenSha256>

<_WasmToolsSha256 Condition="'$(WasmToolHostRid)' == 'linux-x64'">4e2898f7ca3bd0536218ed9b7b36ff7b86954c57ae0e6272fde69728cbe01088</_WasmToolsSha256>
<_WasmToolsSha256 Condition="'$(WasmToolHostRid)' == 'linux-arm64'">4a9a1c6a0885aaf17f433f2927bd61b1fe40069836f750f4e9f9ccd661106c65</_WasmToolsSha256>
<_WasmToolsSha256 Condition="'$(WasmToolHostRid)' == 'osx-x64'">9e311880fe8fcb7ef55cfeea98dc6235440a0aa0a66373affdda4bc1dde47602</_WasmToolsSha256>
<_WasmToolsSha256 Condition="'$(WasmToolHostRid)' == 'osx-arm64'">27b5f83d82ae8f2d961b5a87bdc024bca9db851a7b617d14065cec196fed29d7</_WasmToolsSha256>
</PropertyGroup>

<!-- Composition runs on the build host, not on the wasm target. Use the same
version/host-keyed cache and atomic acquisition as wasi-sdk and wasmtime. -->
<Target Name="AcquireWasiR2RTools">
<Error Condition="'$(_WasiR2RToolOS)' == '' or '$(_WasiR2RToolArch)' == ''"
Text="WASI R2R tool acquisition requires a Linux or macOS x64/arm64 host, not $(WasmToolHostRid)." />
<Error Condition="'$(_BinaryenSha256)' == '' or '$(_WasmToolsSha256)' == ''"
Text="WASI R2R tool acquisition has no SHA-256 hashes for host $(WasmToolHostRid)." />

<MSBuild Projects="$(RepositoryEngineeringDir)wasm\AcquireWasmTool.proj"
Targets="AcquireWasmTool"
Properties="WasmToolName=Binaryen $(BinaryenVersion);
WasmToolUrl=https://github.com/WebAssembly/binaryen/releases/download/version_$(BinaryenVersion)/$(_BinaryenArchiveName).tar.gz;
Comment thread
lewing marked this conversation as resolved.
WasmToolSha256=$(_BinaryenSha256);
WasmToolKind=targz;
WasmToolInnerDir=binaryen-version_$(BinaryenVersion);
WasmToolCacheEntryDir=$(BinaryenCacheDir);
WasmToolStampFile=$(BinaryenStampFile);
WasmToolVerify=$(BinaryenCacheDir)bin/wasm-merge;
WasmToolChmod=$(BinaryenCacheDir)bin/wasm-merge;
WasmToolDownloadDir=$(WasmToolDownloadDir);
HostOS=$(HostOS)" />
<Error Condition="!Exists('$(BinaryenCacheDir)bin/wasm-opt')"
Text="The Binaryen $(BinaryenVersion) cache entry is missing wasm-opt: $(BinaryenCacheDir)" />

<MSBuild Projects="$(RepositoryEngineeringDir)wasm\AcquireWasmTool.proj"
Targets="AcquireWasmTool"
Properties="WasmToolName=wasm-tools $(WasmToolsVersion);
WasmToolUrl=https://github.com/bytecodealliance/wasm-tools/releases/download/v$(WasmToolsVersion)/$(_WasmToolsArchiveName).tar.gz;
WasmToolSha256=$(_WasmToolsSha256);
WasmToolKind=targz;
WasmToolInnerDir=$(_WasmToolsArchiveName);
WasmToolCacheEntryDir=$(WasmToolsCacheDir);
WasmToolStampFile=$(WasmToolsStampFile);
WasmToolVerify=$(WasmToolsCacheDir)wasm-tools;
WasmToolChmod=$(WasmToolsCacheDir)wasm-tools;
WasmToolDownloadDir=$(WasmToolDownloadDir);
HostOS=$(HostOS)" />
</Target>
</Project>
17 changes: 17 additions & 0 deletions eng/wasm/AcquireWasmTool.proj
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
Properties:
WasmToolName (required) name used in messages
WasmToolUrl (required) archive to download
WasmToolSha256 optional expected SHA-256 hash of the downloaded archive
WasmToolKind (required) zip | targz | tarxz | firefox-installer
WasmToolCacheEntryDir (required) final directory, must live under $(WasmToolCacheDir)
WasmToolStampFile (required) sentinel written once the entry is complete
Expand Down Expand Up @@ -67,6 +68,22 @@
<Output TaskParameter="DownloadedFile" PropertyName="_Archive" />
</DownloadFile>

<GetFileHash Condition="'$(_NeedsFetch)' == 'true' and '$(WasmToolSha256)' != ''"
Files="$(_Archive)"
Algorithm="SHA256"
HashEncoding="Hex">
<Output TaskParameter="Items" ItemName="_ArchiveWithHash" />
</GetFileHash>

<PropertyGroup Condition="'$(_NeedsFetch)' == 'true' and '$(WasmToolSha256)' != ''">
<_ActualSha256>%(_ArchiveWithHash.FileHash)</_ActualSha256>
<_NormalizedActualSha256>$(_ActualSha256.ToLowerInvariant())</_NormalizedActualSha256>
<_NormalizedExpectedSha256>$(WasmToolSha256.ToLowerInvariant())</_NormalizedExpectedSha256>
</PropertyGroup>

<Error Condition="'$(_NormalizedActualSha256)' != '$(_NormalizedExpectedSha256)'"
Text="The SHA-256 hash of the downloaded $(WasmToolName) archive does not match the expected value. Expected '$(WasmToolSha256)', actual '$(_ActualSha256)', archive '$(_Archive)'." />

<Unzip Condition="'$(_NeedsFetch)' == 'true' and '$(WasmToolKind)' == 'zip'"
SourceFiles="$(_Archive)"
DestinationFolder="$(_ExtractDir)" />
Expand Down
1 change: 1 addition & 0 deletions eng/wasm/PruneWasmToolCache.proj
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
<!-- Every sentinel this checkout can reference. Browser/JS-engine entries only exist on
platforms where those versions are defined (see wasm-provisioning.targets). -->
<_WasmToolInUseStamp Include="$(EmscriptenSdkStampFile);$(WasiSdkStampFile);$(WasmtimeStampFile)" />
<_WasmToolInUseStamp Include="$(BinaryenStampFile);$(WasmToolsStampFile)" />
<_WasmToolInUseStamp Include="$(ChromeStampFile);$(ChromeDriverStampFile)" Condition="'$(ChromeVersion)' != ''" />
<_WasmToolInUseStamp Include="$(V8StampFile)" Condition="'$(V8Version)' != ''" />
<_WasmToolInUseStamp Include="$(FirefoxStampFile);$(GeckoDriverStampFile)" Condition="'$(FirefoxRevision)' != ''" />
Expand Down
16 changes: 14 additions & 2 deletions eng/wasm/WasmToolCache.props
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<!--
Shared cache for externally acquired WebAssembly build and test tools
(emscripten, wasi-sdk, wasmtime, chrome, chromedriver, firefox, geckodriver, v8).
(emscripten, wasi-sdk, wasmtime, binaryen, wasm-tools, chrome, chromedriver, firefox, geckodriver, v8).

Layout:
$(WasmToolCacheDir)<tool>/<version>-<host rid>/ the tool itself
Expand Down Expand Up @@ -48,7 +48,7 @@
<WasmToolDownloadDir>$([MSBuild]::NormalizeDirectory('$(ArtifactsObjDir)', 'wasm-tool-downloads'))</WasmToolDownloadDir>
</PropertyGroup>

<!-- Tool versions. The version text files are the source of truth: they are also read by the
<!-- SDK and wasmtime version text files are the source of truth: they are also read by the
native build (eng/native/configurecompiler.cmake) and by the native build scripts
(eng/native/gen-buildsys.*), none of which can evaluate MSBuild properties. -->
<PropertyGroup>
Expand All @@ -61,6 +61,12 @@
<WasiSdkVersion Condition="'$(WasiSdkVersion)' == '' and Exists('$(_WasiSdkVersionFile)')">$([System.IO.File]::ReadAllText('$(_WasiSdkVersionFile)').Trim())</WasiSdkVersion>
</PropertyGroup>

<!-- Host tools for the WASI composite-R2R splice pipeline. -->
<PropertyGroup>
<BinaryenVersion Condition="'$(BinaryenVersion)' == ''">130</BinaryenVersion>
<WasmToolsVersion Condition="'$(WasmToolsVersion)' == ''">1.253.0</WasmToolsVersion>
</PropertyGroup>

<!-- Cache entries for the tools whose versions are known this early. Browser and JS engine
entries are computed in eng/testing/wasm-provisioning.targets, once
eng/testing/BrowserVersions.props has been imported. -->
Expand All @@ -73,5 +79,11 @@

<WasmtimeCacheDir>$([MSBuild]::NormalizeDirectory('$(WasmToolCacheDir)', 'wasmtime', '$(WasmtimeVersion)-$(WasmToolHostRid)'))</WasmtimeCacheDir>
<WasmtimeStampFile>$([MSBuild]::NormalizePath('$(WasmToolCacheDir)', 'wasmtime', '$(WasmtimeVersion)-$(WasmToolHostRid).complete'))</WasmtimeStampFile>

<BinaryenCacheDir>$([MSBuild]::NormalizeDirectory('$(WasmToolCacheDir)', 'binaryen', '$(BinaryenVersion)-$(WasmToolHostRid)'))</BinaryenCacheDir>
<BinaryenStampFile>$([MSBuild]::NormalizePath('$(WasmToolCacheDir)', 'binaryen', '$(BinaryenVersion)-$(WasmToolHostRid).complete'))</BinaryenStampFile>

<WasmToolsCacheDir>$([MSBuild]::NormalizeDirectory('$(WasmToolCacheDir)', 'wasm-tools', '$(WasmToolsVersion)-$(WasmToolHostRid)'))</WasmToolsCacheDir>
<WasmToolsStampFile>$([MSBuild]::NormalizePath('$(WasmToolCacheDir)', 'wasm-tools', '$(WasmToolsVersion)-$(WasmToolHostRid).complete'))</WasmToolsStampFile>
</PropertyGroup>
</Project>
Loading
Loading