kernel OOPS at efivar_init with kernel-native

[majst01]

Describe the bug
When booting a fresh installation with kexec, the following OOPS occurs:

[   56.274341] kexec_core: Starting new kernel                                                                                                                                                                     
[    0.000000] Linux version 4.20.4-686.native (mockbuild@kojibld02) (gcc version 8.2.1 20180502 (Clear Linux OS for Intel Architecture)) #1 SMP Thu Jan 24 18:09:21 UTC 2019                                      
[    0.000000] Command line: console=tty0 console=ttyS0 root=PARTUUID=f35c50c2-45a9-4375-842d-c73d5108366b init=/usr/lib/systemd/systemd-bootchart initcall_debug tsc=reliable no_timer_check noreplace-smp rootfst
ype=ext4 net.ifnames=0 biosdevname=0 acpi_rsdp=0x3fbfa014                                                                                                                                                          

...

[    0.385551] BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
[    0.385579] PGD 0 P4D 0
[    0.385579] Oops: 0010 [#1] SMP PTI
[    0.385579] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.20.4-686.native #1
[    0.385579] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015
[    0.385579] RIP: 0010:          (null)
[    0.385579] Code: Bad RIP value.
[    0.385579] RSP: 0000:ffffb542001c7d78 EFLAGS: 00010246
[    0.385579] RAX: 0000000000000000 RBX: ffff8b943d03a400 RCX: 0000000000000000
[    0.385579] RDX: ffffb542001c7dc0 RSI: ffff8b943d03a400 RDI: ffffb542001c7db8
[    0.385579] RBP: ffffb542001c7e30 R08: 0000000000000000 R09: 0000000000000000
[    0.385579] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa2a5fbf2
[    0.385579] R13: ffffb542001c7e48 R14: ffffffffa29f67c5 R15: dead000000000100
[    0.385579] FS:  0000000000000000(0000) GS:ffff8b943f600000(0000) knlGS:0000000000000000
[    0.385579] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.385579] CR2: ffffffffffffffd6 CR3: 000000003b808001 CR4: 00000000003606f0
[    0.385579] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.385579] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    0.385579] Call Trace:
[    0.385579]  ? efivar_init+0xb3/0x390
[    0.385579]  ? vprintk_default+0x1a/0x30
[    0.385579]  ? efivar_ssdt_setup+0x50/0x50
[    0.385579]  ? up+0x2d/0x60
[    0.385579]  ? efivar_ssdt_iter+0xe1/0xe1
[    0.385579]  ? do_early_param+0x94/0x94
[    0.385579]  efisubsys_init+0x132/0x2db
[    0.385579]  ? ktime_get+0x3b/0xb0
[    0.385579]  ? efivar_ssdt_iter+0xe1/0xe1
[    0.385579]  ? do_early_param+0x94/0x94
[    0.385579]  do_one_initcall+0x45/0x1bc
[    0.385579]  ? do_early_param+0x94/0x94
[    0.385579]  kernel_init_freeable+0x1e6/0x290
[    0.385579]  ? rest_init+0xaf/0xaf
[    0.385579]  kernel_init+0x9/0x101
[    0.385579]  ret_from_fork+0x35/0x40
[    0.385579] Modules linked in:
[    0.385579] CR2: 0000000000000000
[    0.385579] ---[ end trace 92830f0b9f2793c4 ]---
[    0.385579] RIP: 0010:          (null)
[    0.385579] Code: Bad RIP value.
[    0.385579] RSP: 0000:ffffb542001c7d78 EFLAGS: 00010246
[    0.385579] RAX: 0000000000000000 RBX: ffff8b943d03a400 RCX: 0000000000000000
[    0.385579] RDX: ffffb542001c7dc0 RSI: ffff8b943d03a400 RDI: ffffb542001c7db8
[    0.385579] RBP: ffffb542001c7e30 R08: 0000000000000000 R09: 0000000000000000
[    0.385579] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa2a5fbf2
[    0.385579] R13: ffffb542001c7e48 R14: ffffffffa29f67c5 R15: dead000000000100
[    0.385579] FS:  0000000000000000(0000) GS:ffff8b943f600000(0000) knlGS:0000000000000000
[    0.385579] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.385579] CR2: ffffffffffffffd6 CR3: 000000003b808001 CR4: 00000000003606f0
[    0.385579] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.385579] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    0.385621] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
[    0.386579] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009 ]---

Expected behavior
Booting up to the login prompt

Screenshots
Environment (please complete the following information):
cat /usr/lib/os-release
NAME="Clear Linux OS"
VERSION=1
ID=clear-linux-os
ID_LIKE=clear-linux-os
VERSION_ID=27640
PRETTY_NAME="Clear Linux OS"
ANSI_COLOR="1;35"
HOME_URL="https://clearlinux.org"
SUPPORT_URL="https://clearlinux.org"
BUG_REPORT_URL="mailto:dev@lists.clearlinux.org"
PRIVACY_POLICY_URL="http://www.intel.com/privacy"

Installed bundles:
swupd bundle-add --no-boot-update
curl
jq
iproute2
openssh-server
package-utils
strace
sudo
kernel-install
kernel-native

My guess is that all clearlinux kernels do not have KEXEC enabled:

grep -i kexec config-4.20.4-native
# CONFIG_KEXEC is not set
# CONFIG_KEXEC_FILE is not set

[ahkok]

I'm not sure that we want to support this for security reasons.
@fenrus75 @miguelinux @busykai comments?

[busykai]

Yes, we don't have kexec enabled for security reasons. It allows non-signed code to be executed and, generally speaking, arbitrary code to be executed. We explicitly disallow that.

@majst01, I'm curious to understand what is the motivation behind the use of kexec to boot in your case?

[majst01]

We are currently building a control plane to create a "metal as a service" platform. In order to have fast end user experience we boot a very small distribution based on u-root via pxe and from there we install the different available linux os. This is done by untar a image the os onto the discs and reboot from there directly into the new installed os with the distribution kernel via kexec. With this method we achieve <1min provision times.

I understand the security impacts, but thought there is a possibility to restrict kexec to signed kernels only: https://lwn.net/Articles/603116/

[busykai]

@majst01, I see, I actually worked on a prototype of similar technology for HPC provisioning. I did use kexec to load production OS as well.

With regards to the existing signature verification mechanism, it only addresses the UEFI Secure Boot scenario (which we don't have enabled by default) and it would only validate the signature with the bootloader. Our concern is more general and in absence of Secure Boot.

You could create your own kernel and your derivative of Clear Linux (with you kernel added to it) as described here: https://github.com/clearlinux/how-to-clear . This approach offers the flexibility of customizing packages in the way you want them to, particularly, to reduce their size and add missing features.

[majst01]

OK understood, thanks for the hint, this is probably the way to go. Havent had a deeper look into, are there any hints on howto provide our own bundle repo server ?

[majst01]

By design because kexec is not enabled in default clearlinux kernels because of security concerns.

[ahkok]

FYI mixer is the tool that generates custom OS content.