Repository navigation
ci: run the pinned jobs on ubuntu-26.04 - #4949
Conversation
|
While independently validating the Gold removal, the original GCC 32-bit sanitizer workload exposed an additional make target-context defect: the parse-time CET probe succeeds for the default 64-bit target and then enforces that result on a 32-bit link. I recorded the actual failing startup-object diagnostics, source fix, real native/compat32 compiler/link/runtime regression, and fully green own-fork CI in #4951. The additional source commits are linked there. Your PR already owns the Gold-removal topic; this is follow-up evidence for the target-specific hardening probe. Codex assisted this investigation and patch. |
496e1c7 to
8a2853e
Compare
libzstd.mk adds -z cet-report=error when the compiler accepts it, but it tests this in 64-bit mode. zstd32, fullbench32, fuzzer32 and zstreamtest32 add -m32 afterwards, and 32-bit system objects usually have no CET properties: with gcc on Ubuntu 24.04 and 26.04, `make -C programs zstd32` fails with "missing IBT and SHSTK properties". The sanitizer targets hid it, because they forced the gold linker, which ignores this option. These targets now link with LDFLAGS_32, which leaves the option out. 64-bit links keep it.
The ASan runtime of gcc-8 loops on AddressSanitizer:DEADLYSIGNAL when the kernel uses 32 bits of mmap randomization. The ubuntu-24.04 image sets vm.mmap_rnd_bits=28; ubuntu-26.04 does not, and gcc-8-asan-ubsan-fuzz printed 102 million such lines there. Both gcc-8 jobs now set it themselves. Newer ASan runtimes (gcc 15, clang) are not affected.
LDFLAGS=-fuse-ld=gold was added in 2017 for Travis CI (f58ac79). The sanitizers link with the default linker, and gold is deprecated in binutils: it is not installed on ubuntu-26.04 runners, so these targets failed there. The 15 sanitizer jobs, including clang-asan-ubsan-cli-tests, return to ubuntu-latest.
qemu-user-static is only a virtual package on ubuntu-26.04, so the ARM and PowerPC install targets failed there. arminstall and ppcinstall now install qemu-user, which exists from Ubuntu 20.04 to 26.04 and on Debian 12, and does not remove an installed qemu-user-static. qemu-user names its binaries qemu-<arch>, without the -static suffix. The targets call qemu-<arch> when qemu-arm is installed, and qemu-<arch>-static otherwise, as before: a machine with only qemu-user-static keeps working. The tests call every cross-compiled binary through QEMU_SYS, so no binfmt registration is needed. armbuild and armfuzz return to ubuntu-latest.
short-tests-1 installed lib64gcc-13-dev-powerpc-cross, which only matches the GCC 13 cross compiler of ubuntu-24.04: with GCC 15 on ubuntu-26.04, ppc64build could not find -lgcc. gcc-multilib-powerpc-linux-gnu installs the 64-bit libraries of the default cross compiler, whatever its version. short-tests-1 returns to ubuntu-latest.
GCC on ubuntu-26.04 runners targets x86-64-v3 by default. With BMI2 enabled, v1.3.1 does not compile: its threading.h defines pthread_mutex_t as a macro, which then clashes with the glibc type pulled in by <immintrin.h>. The versions test now builds with -march=x86-64 on x86-64 hosts. Verified with GCC 15 on Ubuntu 26.04 and x86-64-v3 as default target: the 46 versions build and the test passes. versions-compatibility returns to ubuntu-latest.
The size limits had margins between 1% and 100%. On ubuntu-26.04 runners, GCC 15 targets x86-64-v3 by default, which makes libzstd.so 21% larger (1105472 bytes instead of 909760), above the 1100000 limit. The test now builds with -march=x86-64, so the measure does not depend on the default target of the compiler. Each limit is the size measured with GCC 15.2, plus about 30%, rounded up: this still catches a jump such as #3066. check_size.py prints every size, to make the next measure easy. The workflow documents the rule. libzstd-size returns to ubuntu-latest.
GCC on ubuntu-26.04 runners targets x86-64-v3, which enables BMI2 by default. The dynamic-bmi2 cases without -mbmi2 then still had BMI2, so the generic code path and the runtime dispatch were no longer tested. Each case now starts from -march=x86-64.
8a2853e to
57bed05
Compare
|
Thanks for the investigation @chrisbbreuer and the detailed report in #4951. This PR includes a narrower fix for the same defect: Your approach has a wider goal. We'd rather review that as its own change, once this PR lands. If you open a PR on top of |
The four jobs that test gcc-7 and gcc-8 install them from the focal archive. They also added ppa:ubuntu-toolchain-r/test through make gcc7install and gcc8install, but no package came from it: it only cost about 10 s per job, and a Launchpad SSL error failed one job on meta#4949. The jobs now install the same packages with apt-get directly. The make targets are unchanged, for the machines that still need the PPA.
Follow-up to #4944. This fixes the causes of the ubuntu-26.04 failures and removes the 19 pins to ubuntu-24.04. One commit per cause; each commit removes its own pins:
libzstd.mkadds-z cet-report=errorafter a test in 64-bit mode.zstd32,fullbench32,fuzzer32andzstreamtest32add-m32afterwards, and the 32-bit system objects have no CET properties. With gcc on Ubuntu 24.04 and 26.04,make -C programs zstd32fails with "missing IBT and SHSTK properties". gold ignored this option, which hid the problem in the sanitizer jobs. These targets now link withLDFLAGS_32, without the option; 64-bit links keep it.vm.mmap_rnd_bits=28. The ASan runtime of gcc-8 loops onAddressSanitizer:DEADLYSIGNALwith the 32-bit mmap ASLR of recent kernels: the ubuntu-24.04 image sets 28, ubuntu-26.04 does not.Makefilesanitizer targets no longer force-fuse-ld=gold. It was added in 2017 for Travis CI, and gold is not installed on ubuntu-26.04. 14 jobs, plusclang-asan-ubsan-cli-tests(added by ci: run the cli-tests under ASan and UBSan #4930, pinned to ubuntu-24.04 by b09d9aa).arminstallandppcinstallinstallqemu-userinstead ofqemu-user-static, which is only a virtual package on ubuntu-26.04.qemu-userexists from Ubuntu 20.04 to 26.04 and on Debian 12, and does not remove an installedqemu-user-static. The targets callqemu-<arch>whenqemu-armis installed, andqemu-<arch>-staticotherwise, as before. armbuild, armfuzz.gcc-multilib-powerpc-linux-gnureplaceslib64gcc-13-dev-powerpc-cross, which only matched the GCC 13 cross compiler. short-tests-1.-march=x86-64on x86-64 hosts. With the x86-64-v3 default, v1.3.1 does not compile: itsthreading.hdefinespthread_mutex_tas a macro, which clashes with the glibc type pulled in by<immintrin.h>. versions-compatibility.-march=x86-64, because GCC on ubuntu-26.04 runners targets x86-64-v3 by default, which makeslibzstd.so21% larger;check_size.pynow prints every size;-march=x86-64. With the x86-64-v3 default, the cases without-mbmi2still had BMI2, so the generic code path and the runtime dispatch were no longer tested.Checked locally:
make armbuild, and the ARM fuzzer underqemu-arm.make ppc64buildandmake ppcbuild.make uasan-zstd32, and the 64-bit link still has-z cet-report=error.-march=x86-64brings backDYNAMIC_BMI2=1and BMI2 off.Compatibility, checked in containers on Ubuntu 20.04, 22.04, 24.04 and Debian 12, comparing
devand this PR:make -C programs zstd32with gcc fails ondevon 22.04 and 24.04 (CET), and passes with this PR everywhere;qemu-user-staticinstalled,make armfuzzstill usesqemu-arm-static; aftermake arminstall, it usesqemu-arm, andqemu-user-staticstays installed.Not in this PR:
qemu-consistencyhas been pinned to ubuntu-24.04 since 2025 and installsqemu-user-staticandgcc-14-riscv64-linux-gnu. It needs the same treatment before ubuntu-24.04 is retired.