Repository navigation
Conversation
Android tests run on Helix queues whose machines each boot a single Android emulator from a systemd unit at provisioning time, shared by every work item on that machine. Because the API level is baked into the machine image, covering 16 API levels currently costs 48 dedicated queues and their VM pools, and a new API level costs three more. Move the emulator lifecycle into the work item so that it can come from the test container instead, which lets one queue cover every API level. start-emulator.sh boots the emulator baked into the container image and waits for it to accept an install; stop-emulator.sh shuts it down afterwards. Failures to boot are infrastructure failures rather than test failures, so they request a Helix retry instead of reporting a failed run. The scripts only ever read host and container state, so that a test work item cannot modify the machine it runs on. In particular /dev/kvm is verified but never modified: providing the device is the infrastructure's job. start-emulator.sh detects an emulator that is already running and does nothing in that case, and stop-emulator.sh only stops an emulator that it started itself. The same work item payload therefore works both on the existing queues, where the emulator belongs to the machine and outlives the work item, and on container based queues, where each work item gets its own. The queues can be migrated a few at a time rather than all at once. The wiring is limited to queues that run an emulator. The Windows Android queues drive physically tethered devices and are excluded. Verified by running System.Numerics.Vectors.Tests through XHarness against an emulator booted inside an unprivileged container: 7523 of 7523 tests passed, with the emulator ready 31 seconds after the work item started. Also verified that a second work item reuses a running emulator and leaves it running, and that the payload is attached for a Linux emulator queue but not for a Windows device queue. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 18602fe3-a278-43d2-8cac-2c1909b4758b
Share the emulator payload and pre/post commands between libraries and CoreCLR Helix submissions. Only Android Docker queues receive the hooks; legacy emulator and Windows device queues are unchanged. Add an opt-in runAndroidContainerTests parameter to runtime-androidemulator. It builds CoreCLR and one Directed_r merged JIT APK, then submits that APK to the KVM-enabled queue using the validated, digest-pinned API29 image. The default pipeline continues to use its existing jobs. Validation: bash syntax checks, YAML/XML parser checks, six real MSBuild queue gating cases, and local non-privileged container execution of the CoreCLR smoke app (expected 42) and Directed_r (43 passed, zero failures or skips). The local runtime baseline used the existing upstream argv correction from dotnet#134767. No TLS product fix is included. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 18602fe3-a278-43d2-8cac-2c1909b4758b
|
Azure Pipelines: Successfully started running 6 pipeline(s). 10 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
|
Tagging subscribers to this area: @akoeplinger, @matouskozak, @simonrozsival |
The job template forwards platform, architecture, configuration, runtime and queue parameters to each post-build template. Do not also insert those keys through postBuildStep.parameters: Azure rejects the resulting duplicate keys during template expansion before a build can be queued. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 18602fe3-a278-43d2-8cac-2c1909b4758b
Build and archive System.Numerics.Vectors.Tests in the opt-in container job, then submit it through the libraries Helix path using the same pinned API29 image and KVM-enabled queue as Directed_r. Keep the two test workloads in separate work-item containers; default Android jobs remain unchanged. The first hosted run (1644877) executed 43/43 Directed_r tests successfully inside the container. Validate the other submission path as well rather than relying only on the CoreCLR result. Locally, the exact library build command succeeded and the resulting APK ran 7523/7523 tests, with zero failures/skips, in a fresh non-privileged container. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 18602fe3-a278-43d2-8cac-2c1909b4758b
wfurt
changed the title
This was referenced Oct 22, 2026
Build archived CoreCLR library tests once and submit the same payloads to both API 29 destinations in the opt-in lane. Preserve Directed runtime coverage and leave default pipeline jobs unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 18602fe3-a278-43d2-8cac-2c1909b4758b
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prototype a per-work-item Android emulator using the published API 29 image from
dotnet/dotnet-buildtools-prereqs-docker#1745 and the separately landed KVM queue
support.
This is an infrastructure experiment only. It contains no TLS product fix.
eng/testing/Android.submissions through
eng/testing/Android/helix.targets.Windows device queues unchanged.
runAndroidContainerTestsparameter to perform API 29 parityvalidation. Build CoreCLR and the library test archives once, then submit the
same payloads to both
Ubuntu.2204.Amd64.Android.29.Openand the KVM-enabledqueue with the digest-pinned API 29 container image.
JIT/Directed/Directed_r.csprojruntime coverage and replace theNumerics-only library selection with the standard inner-loop library test
archives, including networking, IO and globalization. Default jobs are unchanged.
Validation
Parity extension
false, one archived library build feeds two API 29 queues, and both normal
runtime/library submission paths remain in place.
runAndroidContainerTests=true: passed. Expandedto the Helix monitor and one build job; verified both queue lists and
TestScope=innerloopin the generated submission steps.runAndroidContainerTests=false: passed. Retainedthe legacy CoreCLR/NativeAOT jobs and excluded the parity job.
receives one payload, pre-command and post-command; legacy Android, Windows
containers, non-Android and unset queues receive none.
bash -n eng/testing/Android/start-emulator.sh eng/testing/Android/stop-emulator.sh:passed.
git diff --check: passed.The expanded library build and paired test execution have not yet completed.
No new local product build was performed for this YAML/documentation extension.
The first paired run is build 1651478.
Earlier execution proof
succeeded.
Directed_r: 43 passed, 0 failed, 0 skipped.System.Numerics.Vectors.Tests: 7,571 passed, 0 failed, 0 skipped in theuploaded test XML. Both work items logged Docker execution, emulator startup,
actual test execution and teardown.
Directed_r: 43 passed, 0 failed, 0 skipped../src/tests/build.sh -os android x64 Release -test JIT/Directed/Directed_r.csproj:passed on the retained local baseline with the upstream launcher correction
from [Apple][Android][CoreCLR] Don't pass the program path as managed args[0]; quarantine newly running runtime tests #134767. Fresh non-privileged API 29 container:
43 passed, 0 failed, 0 skipped.
./dotnet.sh build src/libraries/System.Numerics.Vectors/tests/System.Numerics.Vectors.Tests.csproj -c Release /p:TargetOS=android /p:TargetArchitecture=x64 /p:RuntimeFlavor=CoreCLR /p:ArchiveTests=true:passed. Fresh non-privileged local container: 7,523 passed, 0 failed, 0 skipped
on the older retained source baseline.
42.
Helix's existing privileged Docker mode is unchanged; non-privileged execution
was verified locally, not in hosted Helix.
CI scope and remaining gaps
Only the opt-in parameter selects parity validation. It requests one CoreCLR x64
Release build and submits the Directed runtime APK plus the normal inner-loop
library payloads to both API 29 destinations. Each Docker work item has a fresh
container and emulator; the legacy queue retains its host-managed emulator.
Helix job names identify destination queues, and library runs use
CoreCLR_Release_API29_Parity_Libraries. Compare individual test identities,failures, skips, timeouts and durations, not just aggregate green status.
Result comparison is not automated by this pipeline.
This does not yet validate the full CoreCLR runtime suite, NativeAOT, Mono, other
API levels or architectures. API-level equality alone does not establish exact
system-image/emulator-version parity. No duration or reliability improvement is
claimed. Earlier logs also expose ignored IPv6
-proparguments and a missingSetStressModes_normal.shdiagnostic; neither has yet been classified throughmatched legacy/container execution. No guest-root or crash-dump provisioning is
part of this path.
Note
This pull request description was generated with GitHub Copilot.