Skip to content

aarch64: give direct-booted guests ACPI via a synthesized EFI handoff - #3

Closed
tonicmuroq wants to merge 1 commit into
devfrom
feat/aarch64-acpi-direct-boot-pr
Closed

tonicmuroq wants to merge 1 commit into
devfrom
feat/aarch64-acpi-direct-boot-pr

Conversation

@tonicmuroq

@tonicmuroq tonicmuroq commented Sep 19, 2026

Copy link
Copy Markdown

Why

A direct-kernel-booted aarch64 guest only ever saw the FDT. Every ACPI table create_acpi_tables() wrote went unused, and since CH's PCI hotplug is ACPI-only (GED ESCNPHPR.PSCNPCNTDVNT/B0EJ), vm.add-net and vm.remove-device both returned success while the guest noticed neither:

_virtio-pci-_net3                    -> 0000:00:05.0   old NIC, never ejected
_virtio-pci-cocoon-net-4ab3d9e2bee8  -> 0000:00:0a.0   new NIC, invisible to the guest

For anything that clones from a snapshot this means the clone keeps the snapshot's MAC, so a MAC-keyed DHCP server hands every clone the same address, and the throwaway restore tap leaks for the life of the VM. x86 is unaffected — direct kernel boot still carries ACPI there.

What

The arm64 kernel doesn't need real firmware for ACPI. It takes the RSDP from the EFI configuration table, which it finds via linux,uefi-system-table and the linux,uefi-mmap-* pointers in /chosen, and it doesn't care whether firmware or the VMM produced those structures. So we synthesize them. (Same approach OpenVMM uses for aarch64 direct boot.)

  • arch/src/aarch64/efi.rs (new) — EFI System Table, configuration table, and EFI memory map. Dependency-free, including a small CRC32.
  • fdt::create_stub_fdt() — a device tree with only /chosen. dt_scan_depth1_nodes() treats any other depth-1 node as proof of a real DTB and disables ACPI, so describing no hardware is exactly what lets the guest take its hardware from ACPI — and is why acpi=force is not needed.
  • Selected by --platform acpi_boot=on|off, defaulting on for aarch64.

Four configuration tables are published:

GUID why
ACPI 2.0 the RSDP — the point of the exercise
EFI RT Properties declares no runtime services, so the guest never calls into firmware that isn't there
SMBIOS3 aarch64 Linux locates DMI only through this entry, so the tables setup_smbios() writes were unreachable on the device-tree path and are now visible
LINUX_EFI_MEMRESERVE lets the GICv3 ITS persist its LPI property/pending tables through efi_mem_reserve_persistent(); without it the guest warns twice per boot out of irq-gic-v3-its.c

linux,uefi-secure-boot = 0 is load-bearing. Ubuntu's kernel makes it a required property in efi_get_fdt_params(). Without it the EFI handoff is abandoned, the memory map is never installed, and — with no /memory node — memblock comes up empty and paging_init panics with Failed to allocate page table page. The symptom looks nothing like secure boot. Mainline ignores the property.

Verification

On a c4a-highmem-96-metal host (aarch64 bare metal), Ubuntu 24.04 arm64 guest, direct kernel boot:

efi: EFI v2.7 by Cloud Hypervisor
efi: ACPI 2.0=0x40200000 RTPROP=0x401010d0 SMBIOS=0x40400000 MEMRESERVE=0x40102000
secureboot: Secure boot could not be determined (mode 0)
ACPI: RSDP/XSDT/FACP/DSDT/APIC/PPTT/GTDT/MCFG/SPCR/DBG2/IORT
psci: probing for conduit method from ACPI.
gic_acpi_init -> GICv3: 224 SPIs, ITS
acpiphp: Slot [0..31] registered
hw perfevents: enabled with armv8_pmuv3_0 PMU driver, 21 counters available
check result
/sys/firmware/ acpi devicetree dmi efi fdt — devicetree holds only the stub
GED ACPI0013:00 bound, 12: GICv3 42 Edge ACPI:Ged
kernel warnings 0
memory map three contiguous ranges, no unavailable ranges
PMU armv8_pmuv3_0, 21 counters (MADT GICC performance_interrupt = 0x17)
DMI sys_vendor = Cloud Hypervisor
vm.add-net GED irq 0→1, guest gets the NIC with no rescan
vm.remove-device GED irq 1→2, guest ejects it, it leaves CH's device tree
clone new MAC appears by itself, DHCPs a distinct lease, snapshot NIC gone, no leftover restore tap
acpi_boot=off same guest still boots through the device tree, 0 ACPI tables, 0 warnings

Unit tests

Nine new tests, all on aarch64. The byte-level structures and the two invariants the whole mode rests on are the parts that would fail silently, so those are what they cover:

test what breaks without it
system_table_header_is_well_formed signature/revision/header_size, the CRC32 recomputed over the zeroed header, and the UTF-16LE vendor string
configuration_table_points_at_every_structure all four GUIDs and their targets; RT Properties really declaring nothing supported; the memreserve header starting empty with capacity
memory_map_covers_ram_without_overlapping any gap or overlap (either makes the kernel reject the map, and the stub tree has no /memory to fall back on), plus firmware-owned spans never being typed conventional and the EFI region being runtime- not boot-services data
stub_fdt_has_only_a_chosen_node a second depth-1 node, which sends dt_scan_depth1_nodes() back to "real DTB" and turns ACPI off
stub_fdt_publishes_the_efi_handoff any linux,uefi-* property, including linux,uefi-secure-boot
stub_fdt_carries_the_initramfs_when_there_is_one initrd start/end
metadata_stays_below_the_acpi_region, crc32_matches_known_vector, reserved_span_is_punched_out_of_ram layout overflow, the CRC implementation, the range subtraction

Mutation-checked — each of these fails on the corresponding break and nothing else does:

delete linux,uefi-secure-boot      -> stub_fdt_publishes_the_efi_handoff        FAILED
add a depth-1 node to the stub DT  -> stub_fdt_has_only_a_chosen_node           FAILED
EFI region back to BOOT_SERVICES   -> memory_map_covers_ram_without_overlapping FAILED

cargo fmt --check, cargo clippy --release --all-targets and cargo test -p arch are clean on aarch64.

Open questions

  1. Defaulting on is a judgement call. It's what makes this useful without every caller changing, but it switches hardware discovery for all aarch64 guests. Happy to flip it to opt-in if you'd rather land it dark first.
  2. Snapshot/restore of the EFI structures is only covered indirectly. They live in guest RAM so they ride along with the memory snapshot, and clone was tested end to end, but there's no dedicated test.
  3. configure_system_acpi() ignores vcpu_topology, pci_space_info and the vgic handle — all of that is ACPI's job now. It's the right split, but the two paths do take visibly different inputs.
  4. Multi-NUMA is unverified. Single-node behaviour is identical to the device-tree path (both fall back to a faked node, since CH emits no SRAT for one node); SRAT/SLIT handling under --platform num_pci_segments/NUMA configs hasn't been exercised.

Note on the branch

Based on origin/dev. My local dev had three unpushed commits, so this was cherry-picked onto the remote head to keep the diff to this change alone.

@tonicmuroq
tonicmuroq force-pushed the feat/aarch64-acpi-direct-boot-pr branch from 8e705ef to 29e17dd Compare September 19, 2026 14:47
A direct-kernel-booted aarch64 guest only ever saw the FDT, so every ACPI
table create_acpi_tables() had already written went unused. PCI hotplug is
ACPI-only (GED -> PHPR.PSCN -> PCNT -> DVNT/B0EJ), which left vm.add-net and
vm.remove-device returning success while the guest noticed neither: a hot-added
NIC never appeared, and an ejected one stayed on the bus forever along with its
tap.

The arm64 kernel does not need real firmware for this. It takes the RSDP from
the EFI configuration table, which it finds through `linux,uefi-system-table`
and the `linux,uefi-mmap-*` pointers in the device tree's /chosen node, and it
does not care who produced those structures. So synthesize them:

  - arch/src/aarch64/efi.rs builds an EFI System Table, a configuration table
    and an EFI memory map.
  - fdt::create_stub_fdt() emits a device tree holding only /chosen.
    dt_scan_depth1_nodes() treats any other depth-1 node as proof of a real
    DTB and disables ACPI, so describing no hardware is what lets the guest
    take its hardware from ACPI — and is why acpi=force is not needed.

Selected by `--platform acpi_boot=on|off`, defaulting on for aarch64.

The stub tree also carries `linux,uefi-secure-boot = 0`: Ubuntu's kernel makes
it a required property in efi_get_fdt_params(), and without it the EFI handoff
is abandoned, the memory map is never installed, and — with no /memory node —
memblock comes up empty and paging_init panics with "Failed to allocate page
table page". Mainline ignores the property.

Four configuration tables are published. ACPI 2.0 carries the RSDP. EFI RT
Properties declares no runtime services, so the guest never calls into
firmware that does not exist. SMBIOS3 is how aarch64 Linux locates DMI at all,
which makes the tables setup_smbios() writes reachable for the first time on
this path. LINUX_EFI_MEMRESERVE lets the GICv3 ITS persist its LPI property
and pending tables through efi_mem_reserve_persistent(); without it the guest
warns twice per boot out of irq-gic-v3-its.c.

Verified on a c4a-highmem-96-metal host, Ubuntu 24.04 arm64 guest:

  - /sys/firmware/{acpi,efi,dmi} present, ACPI0013 GED bound to a GIC SPI,
    GICv3/ITS, PSCI, PMU, the SPCR console and DMI all taken from ACPI
  - zero kernel warnings, and no unavailable ranges in the memory map
  - vm.add-net: GED interrupt fires, the NIC appears with no guest-side rescan
  - vm.remove-device: the guest ejects it and it leaves CH's device tree
  - clone: the new MAC appears by itself and DHCPs a distinct lease, the
    snapshot's NIC is gone, and no restore tap is left behind
  - acpi_boot=off still boots the same guest through the device tree
@tonicmuroq
tonicmuroq force-pushed the feat/aarch64-acpi-direct-boot-pr branch from 29e17dd to 8108d55 Compare September 19, 2026 14:58
@CMGS

CMGS commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator

Closing in favour of the patch-branch flow we use for everything that is headed upstream: a single commit on a branch rebased onto upstream/main, rather than a PR against dev.

The work is carried over unchanged onto aarch64-acpi-direct-boot, rebased onto upstream/main (SMBIOS layout landed upstream in cloud-hypervisor#8786, so it applies cleanly), plus the review findings:

  • SMBIOS region typed EFI_RUNTIME_SERVICES_DATA instead of EFI_BOOT_SERVICES_DATAis_usable_memory() hands boot-services data back to memblock as System RAM, and arm64 scans DMI from arm_dmi_init(), a core_initcall, by which point the buddy allocator is live.
  • A dedicated error for a missing RSDP instead of reusing SetupFdt.
  • A bound on the memory map so it cannot run past its page into the system table.
  • EFI_MAX_SIZE actually used.

Nothing about the approach changed — the analysis and the verification in this PR still stand.

@CMGS CMGS closed this Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants