feat(profile): zero-cost-when-off fast path; default MI_PPROF=1 in headers

[zackees]

Closes #267.

Update: addressed Opus review (3 items)

HIGH (fixed, commit 769adbf): mi_theap_queue_first_update (page-queue.c) used to
early-return while prof_force_slow was set, freezing pages_free_direct instead of
updating it. That's unsafe: a page retired (_mi_page_free -> mi_page_queue_remove ->
this function) while poisoned could leave a stale pointer in pages_free_direct after the
arena reclaims its memory -- a use-after-free on the next fast-path allocation of that size
class. Fixed by substituting the empty page for the real one instead of returning early, so
every call keeps writing a value consistent with "currently valid queue page, or empty".

Reproduced the race window itself with a clang ThreadSanitizer build (-DMI_DEBUG_TSAN=ON -DCMAKE_C_COMPILER=clang -DMI_BUILD_SHARED=ON): with the old early-return code reinstated,
TSan reliably (repeatedly, across a single run) flags the racy read of prof_force_slow
gating that early return in mi_theap_queue_first_update, racing
_mi_subproc_prof_set_force_slow's write. With the fix, TSan still flags the same
underlying (by-design, documented, non-atomic) flag read/write -- expected, and unchanged
in character -- but it no longer gates skipping a pages_free_direct write. Plain ASan
(gcc -fsanitize=address) and 30 plain-debug runs of the reverted build did not catch
memory corruption directly within the test's short runtime -- plausible since actually
observing the UAF needs the arena to decommit/repurpose the freed page's memory, which
purge_delay (default ~1s) likely didn't trigger before the short-lived test process
exited. TSan is the tool that actually demonstrates this race is reachable, not a
theoretical concern.

MEDIUM (fixed, commit 769adbf): _mi_subproc_prof_set_force_slow took the target
state as a bool enable snapshot captured before its walk began. Two racing start/stop
calls could have their walks finish out of order, leaving every theap poisoned while
disabled or vice versa. Fixed by dropping the parameter and reading mi_prof_is_enabled()
fresh per theap, inside the same heap->theaps_lock a concurrently-created theap uses to
read the same flag.

MEDIUM (fixed, commit 769adbf): the adversarial test's workers now allocate/free in
batches of 256 across 3 size classes (so pages actually retire during the race, which the
HIGH fix's exercise depends on) and assert -- via accum-mode sample counts snapshotted with
the main thread idle -- that worker-thread allocations are sampled while the profiler runs,
not just the main thread's own canary allocations.

LOW (done):

• ci/verify_local.py's lint config now uses pyright[nodejs]==1.1.411 (commit
  2468ad9) so it resolves offline instead of trying to download a Node.js runtime.
• prof_auto_start() now documents a latent (not fixed here) self-deadlock hazard: a
  mi_subproc_visit_heaps visitor callback that allocates for the first time would
  re-enter subproc->heaps_lock non-reentrantly via the new walker. Left as a follow-up.
• Disassembly correction: "byte-for-byte identical" in the original PR description
  applies to mi_malloc/mi_heap_malloc_small only. mi_free's fast local-free path
  gains one extra cmpb $0x0,0x78(%rbx) / jne (the hoisted page->has_metadata check)
  compared to MI_PPROF=OFF -- confirmed by disassembling both Release builds' mi_free
  (aliased to _ZdaPv/operator delete[]): the OFF build's fast local-free path goes
  straight from the double-free check to _mi_memevt_on_free; the ON build inserts the
  has_metadata compare-and-branch first. This is the intentional, minimal cost of the
  free-side fix (avoiding an out-of-line call on every free of an unsampled page); it does
  not affect the mi_malloc disassembly claim, which remains byte-identical.

verify_local now also has lint green; rust re-passes after the trailing amalgamation
regen commit. off/release/debug-full/guarded/shared/memory-gate/diag all
still PASS. asan remains an environmental FAIL (unrelated toolchain gap, unchanged from
before) -- see prior comment. test-profile-race soaked 20x with zero failures.

---

Summary

Ports Bun's (oven-sh/mimalloc @ 942b8342, MIT) prof_force_slow strategy so that
mi_malloc/mi_heap_malloc_small's fast path contains zero profiler instructions
when the profiler is stopped or compiled out, fixing the +70% ns/alloc regression
measured in #154. Also defaults MI_PPROF to 1 in the header, matching CMake's own
default, per the issue's Step 5 policy decision.

Design

• mi_theap_t gains prof_force_slow (include/mimalloc/types.h, #if MI_PPROF).
  While set, mi_theap_queue_first_update (src/page-queue.c) leaves
  pages_free_direct poisoned (pointing at the static empty page) instead of
  publishing a real free page, so every malloc misses the fast list-pop and lands in
  _mi_malloc_generic, where the sampling countdown now lives exclusively (moved out
  of alloc.c:mi_page_malloc_zero, which stays shared with the fast path and now has
  zero profiler code).
• mi_prof_start/mi_prof_stop (src/profile.c) call a new cross-theap walker,
  _mi_subproc_prof_set_force_slow (src/subproc.c), outside prof_lock, to flip
  the flag process-wide (mi_subprocs -> heap->next -> theap->hnext, each level under
  its existing lock: mi_subprocs_lock -> subproc->heaps_lock -> heap->theaps_lock).
  • Start also poisons pages_free_direct directly
    (_mi_theap_pages_free_direct_poison, page-queue.c).
  • Stop deliberately does not touch pages_free_direct cross-thread — reading
    another theap's pq->first while that theap's own thread concurrently mutates its
    queues (e.g. _mi_page_free removing that very page) can publish a page that's
    freed moments later, a same-thread-vs-cross-thread use-after-free. Each theap
    re-syncs its own pages_free_direct, same-thread, lock-free, the next time it takes
    the generic path (mi_find_page/_mi_malloc_generic in page.c).
• A theap created concurrently with start/stop reads mi_prof_is_enabled() under the
  same heap->theaps_lock the walker holds while visiting that heap's list
  (theap.c), so a racing theap creation is never missed by the walker.
• _mi_prof_on_alloc now checks _mi_meta_is_meta_page() before prof_auto_start()
  (previously after): a meta allocation (_mi_meta_zalloc, used to bootstrap a thread's
  own tld/theap) can hold subproc->theap_meta_lock or subproc->heaps_lock, and the
  new walker needs those same locks — reachable only from a genuine first user
  allocation now, never from meta bootstrap (would otherwise be a same-thread
  non-reentrant-lock deadlock).
• Free path (free.c, page.c): hoisted the existing page->has_metadata check to the
  call site, before the (previously unconditional) out-of-line _mi_prof_on_free/
  _mi_prof_on_free_collect call, avoiding a function-call on every free of an unsampled
  page.
• include/mimalloc/types.h: #ifndef MI_PPROF #define MI_PPROF 1 #endif. CMake keeps
  passing -DMI_PPROF=0/1 explicitly, which still wins (command-line define beats the
  in-header default). README's non-CMake-consumer paragraph updated to match.

Every ported block carries // #267: ... provenance comments; the strategy (not the
code) is imported from oven-sh/mimalloc @ 942b8342, MIT.

Numbers

Disassembly (primary evidence, noise-free): mi_malloc/mi_heap_malloc_small in
the Release static lib (libmimalloc.a, MI_PPROF=ON) contain no call to any
_mi_prof_* function and no load of profiler state — the compiled bytes are
byte-for-byte identical to the same functions built with MI_PPROF=OFF (only the base
address differs). mi_heap_malloc_small's only call is _mi_memevt_on_alloc (the
always-on, accepted memory-events hook), and its slow-path branch jumps straight to
_mi_malloc_generic:

0000000000001590 <mi_heap_malloc_small>:
    ...
    15c8: je     15f0 <mi_heap_malloc_small+0x60>   ; block == NULL -> slow path
    ...
    15e2: call   15e7 <...>                          ; _mi_memevt_on_alloc (accepted, always-on)
    15ef: ret
    15f0: ...
    15fc: jmp    1601 <...>                           ; _mi_malloc_generic (poisoned fast-path miss)

Header default verified both directions with a bare compile (issue's exact acceptance
test):

$ c++ -x c++ -c src/static.c -Iinclude -DMI_STATIC_LIB -o /tmp/o.o   # no MI_PPROF define
$ nm /tmp/o.o | grep 'T mi_prof_start$'
000000000002c13a T mi_prof_start        # real symbol, not the stub -> default is ON
$ c++ -x c++ -c src/static.c -Iinclude -DMI_STATIC_LIB -DMI_PPROF=0 -o /tmp/o2.o
$ objdump -d /tmp/o2.o | grep -A3 '<mi_prof_start>:'   # -> immediately returns 0 (the stub)

Timing (corroborating, noisy shared dev container — not a dedicated benchmark host,
unlike #154's Windows/MinGW reference machine — direction is unambiguous and the
disassembly proof above is authoritative):

build	ns per alloc+free pair (best of 5, N=4M, single-threaded)
MI_PPROF=OFF	~24–31
MI_PPROF=ON, profiler stopped	~28
MI_PPROF=ON, profiler running (default rate)	~33–34

MI_PPROF=ON-stopped is within noise of MI_PPROF=OFF (down from #154's measured
+70%); the actively-running cost is paid only while a profile is in progress. Full
writeup in docs/profiler.md (force-added; docs/ is gitignored in this repo).

Tests

• Extended test/test-profile.c with the issue's adversarial checklist in one new case
  (test_start_while_allocating_stop_mid_sample_restart_reset): start while worker
  threads are mid-allocation-loop, stop mid-sample, restart with a different rate while
  workers race the restart too, and mi_prof_reset while a page still carries a live
  sampled block.
• All existing profiler/DHAT/memory-events tests green.
• test-profile-race and test-subproc-lifecycle run directly (outside
  verify_local's default fast tier) on Debug-full+guarded (MI_PPROF=ON),
  MI_PPROF=OFF, and a clang ASan build — all green, ASan reports no errors (no
  use-after-free, no leak) across 5 repeated runs of the new adversarial test plus the
  full suite.

verify_local table

config      | result | seconds
------------+--------+--------
release     | PASS   | 73.0
off         | PASS   | 63.0
debug-full  | PASS   | 105.4
guarded     | PASS   | 150.4
shared      | PASS   | 36.9
memory-gate | PASS   | 69.8
diag        | PASS   | 75.8
rust        | PASS   | 112.7   (re-run after regenerating the vendor amalgamation)
lint        | FAIL   | 74.8    -- environmental: pyright's bundled pyright needs to
                                   download a nodeenv on first run; this sandbox has no
                                   outbound network/CA trust for that download (SSL cert
                                   verify failure). ruff check/format and every other
                                   lint-job step (benchmark-policy selftests, 81 pytest
                                   cases) pass.
asan        | FAIL   | 0.1     -- environmental: CMake's sanitizer/asan_interface.h probe
                                   uses the `clang` on PATH
                                   (/run/current-system/sw/bin/clang), which has no
                                   bundled sanitizer headers in this sandbox (a *different*
                                   vendored clang toolchain elsewhere on the box does).
                                   Verified ASan-clean independently with gcc's
                                   -fsanitize=address instead: full ctest suite plus the
                                   new adversarial test, 0 findings.

gh issue comment 10 posted with the same numbers.

What the issue got wrong / diverged from

• mi_prof_reset does exist (issue text implied it might not, in the "map that
  adversarial case to stop+restart" fallback instruction) — it only resets the accum
  counters and interned-stack table, not live records, so the reset-while-flagged test
  case verifies the (pre-existing, unaffected-by-this-PR) live-block behavior rather than
  anything Bun-specific about "keeping the flag."
• ci/dev_linux.py bench requires Docker orchestration out of scope for a quick
  before/after; used a small standalone microbenchmark matching docs: measure and publish the MI_PPROF=ON disabled-path overhead (#50) #154's methodology
  instead (alloc+free pairs, best-of-5, interleaved). --only bundle from the task brief
  isn't a verify_local.py --list target at this pin — noted, not treated as a gate.

🤖 Generated with Claude Code

https://claude.ai/code/session_01YT4jVokb2gdT8ngFrH8i8S

[zackees]

Opus review rounds 1–3: APPROVE after fixes. Round 1 found a real UAF window (early-return in mi_theap_queue_first_update suspended the pages_free_direct invariant; fixed by substituting the empty page like Bun), a start/stop walk ordering race (walker now reads mi_prof_is_enabled() per theap), and a test that could not reach page retirement (now batch-allocates). Round 2 found the range short-circuit still proxying an interior slot under the cross-thread poisoner (fixed: short-circuit disabled while poisoned). Verified: mi_malloc/mi_heap_malloc_small instruction-identical ON vs OFF; ON-stopped within 1–2% of OFF (23.5–24.0 vs 23.5–25.3 ns); all profiler/DHAT/memevt tests green on debug-full and guarded; amalgamation drift clean.

Merging: the two reds are memory-gate (ubuntu-latest) (baseline drift, deferred per #276) and test-win-gnu (#236 timed_allocation_audit flake).