Ran the interrupt handler from ATCM, and measured what that buys - #630
Merged
fdesbiens merged 1 commit intoAug 17, 2026
Merged
Conversation
The TCM work so far enabled ATCM and left it empty, which buys nothing.
This places code in it and measures the result.
link.lds gains an ATCM region and an .atcm_text section whose run address
is in the bank and whose load address is in CODE. tcm_copy_atcm_text
moves it, using 64-bit stores because ECC is enabled on this part and
ATCM requires them (Cortex-R52 TRM 6.2.2); both ends of the section are
8-byte aligned so there is no narrower tail to leave without check bits.
The copy runs after T4 and T5, which write test patterns to the first and
last words of the bank and would otherwise land on top of the code.
s32z280_atcm.elf is the same image as s32z280_boot.elf with the interrupt
service body placed in ATCM. Both targets exist so the comparison can be
repeated on one board in one session without reconfiguring. The service
routine is split into a timed wrapper that stays in .text and a body that
moves, so the wrapper's own cost appears in both measurements and cancels.
Measured in PMU cycles over 64 samples, caches enabled in both:
code RAM ATCM change
min 454 334 -26.4%
mean 458 340 -25.8%
max 612 466 -23.9%
spread 158 132 -16.5%
CNTPCT is not used for this: at 8 MHz it cannot resolve a handler body,
let alone the variation in one.
The level shift is the solid part. ATCM is a quarter faster even though
the caches were on and code RAM had the instruction cache available,
which says the handler does not stay resident between interrupts 10 ms
apart -- so each one pays a cold fetch from code RAM, which runs at half
the core frequency where ATCM runs at full speed with one wait state
(S32Z2 RM 6.3.6).
The determinism claim deserves less weight than the numbers first
suggest. The spread narrows by only 16%, and ATCM's worst case still sits
slightly above code RAM's best case, so the two distributions overlap at
the tails rather than separating. Whatever jitter remains is not
dominated by instruction fetch.
Both images pass six of six boot probes.
Assisted-by: Claude Code (Opus 5) <noreply@anthropic.com>
This was referenced Aug 17, 2026
fdesbiens
added a commit
that referenced
this pull request
Aug 17, 2026
…lds (#632) The handler comparison in #630 measured one alignment, which is the mistake that made the cache benchmark in this example report 24% or 0% for identical silicon. The handler body is now generated at four offsets within a cache line, all four are measured, and the figures are reported per placement. The claim survives. Mean cycles for the handler body: loop offset code RAM ATCM 0 523 383 16 534 377 32 539 375 48 521 383 ATCM is faster at every placement, by about 28%, and the two sets of means do not overlap. Worst case improves as well, 510 against 694. Two things worth recording beyond the headline. The handler measurement is only mildly alignment sensitive, 3.5% across placements in code RAM and 2% in ATCM, quite unlike the cache loop's two modes. So this comparison was less fragile than the cache one, and #630's direction was right even though its method was not defensible. The absolute numbers differ from #630 because the body now sits behind a placement wrapper that adds a call; the comparison is internally consistent either way. Both variants also report identical cache sweeps, 0 and 0 and 240 and 240, which settles the regression this branch's predecessor appeared to show. That apparent regression was the single-alignment probe moving between its two modes, not anything about ATCM. One copy of the logic is kept: the wrappers inline a single always_inline implementation, so the four placements cannot drift apart. Assisted-by: Claude Code (Opus 5) <noreply@anthropic.com>
fdesbiens
added a commit
to fdesbiens/threadx-fd
that referenced
this pull request
Aug 17, 2026
The code side of TCM was done in eclipse-threadx#630; this is the data side. One of the demo's three thread stacks now lives in BTCM and the other two stay in DRAM0, so a run exercises both paths and a mistake in either shows up. link.lds gains a BTCM region and a .btcm_bss NOLOAD section, so a stack is an ordinary C array with a section attribute and the linker checks it fits, rather than a hardcoded address that silently overflows the bank. entry.S preloads the whole bank at EL2, and that is not optional. ECC is enabled on this part, so a TCM location must be written before it can be read (TRM 6.2.2), and a stack is read before the program writes it -- the first context restore pops what tx_thread_create built into it. The preload has to happen before any C runs, because the demo images do not run bsp_boot.c, which is where the ATCM preload lives. 32-bit stores suffice for BTCM where ATCM needs 64-bit. Why BTCM for a stack: 16 KB at zero wait states where ATCM has one, and never cached whatever the MPU says about it. Measured in the previous commit, uncached BTCM comes within 6.6% of the best cached case while uncached DRAM0 is 22% off it, so stack access runs at close to cache-hit speed without depending on a line being resident. That is the property a determinism argument needs. What this commit does not claim: no thread-level timing improvement has been measured. The case for BTCM here rests on the memory characterisation and on removing the cache from the path, not on a measured context-switch figure. That measurement is worth doing and has not been done. Verified on the S32Z280-594EVB. The demo reports its stack addresses so the placement is visible rather than implied -- sleeper at 0x30100000 in BTCM, spinner and judge in DRAM0 -- and passes with 100 ticks, 20 sleeper wakeups and 20 preemptions, so a real thread schedules, preempts and context-switches on a tightly-coupled-memory stack. The boot image still passes six of six probes. Assisted-by: Claude Code (Opus 5) <noreply@anthropic.com>
fdesbiens
added a commit
that referenced
this pull request
Aug 17, 2026
* Enabled BTCM and measured what it offers as a data store
BTCM was disabled because of a regression that turned out not to exist; the
claim was retracted in the previous commit. It is enabled now, and this
measures why that is worth doing: 16 KB at zero wait states, where ATCM has
one, and no cache in the path at all.
Enabling it needs three things, all of which existed for ATCM already: the
region register write at EL2, an MPU region, and an ECC preload before any read
(TRM 6.2.2). BTCM accepts 32-bit stores where ATCM needs 64-bit, which
tcm_preload already handles.
The cache benchmark now sweeps three memories rather than one, four loop
alignments each. At the alignments where the loop is not instruction-fetch
bound:
memory cold (uncached) warm (cached) gain
DRAM2 half-speed 857,540 651,436 24.0%
DRAM0 full-speed 797,824 651,297 18.3%
BTCM zero wait 694,689 651,369 6.2%
Three things follow.
Warm times are identical across all three memories, within 0.02%. Once the data
cache is working the backing store barely matters, because the working set fits
in it.
Cold times rank as the reference manual predicts: BTCM fastest, then DRAM0,
then DRAM2 at half the core frequency (S32Z2 RM 6.3.6).
BTCM still shows a 6.2% gain when the caches are enabled, and that cannot be
the data cache, because an enabled TCM is Non-cacheable Non-shareable Normal
memory whatever the MPU says. It is the instruction cache on the timing loop.
This probe has always measured both caches together; three memories side by
side is what makes that visible.
The number that matters for placing data in BTCM: uncached BTCM is within 6.6%
of the best cached case, where uncached DRAM0 is 22% off it. Data in BTCM runs
at close to cache-hit speed with no cache to miss, which is the determinism
argument stated as a measurement rather than an assertion.
DRAM2's sweep is unchanged with BTCM enabled, 0 and 0 and 240 and 240, which
independently confirms the retraction.
Assisted-by: Claude Code (Opus 5) <noreply@anthropic.com>
* Put a ThreadX thread stack in BTCM
The code side of TCM was done in #630; this is the data side. One of the demo's
three thread stacks now lives in BTCM and the other two stay in DRAM0, so a run
exercises both paths and a mistake in either shows up.
link.lds gains a BTCM region and a .btcm_bss NOLOAD section, so a stack is an
ordinary C array with a section attribute and the linker checks it fits, rather
than a hardcoded address that silently overflows the bank.
entry.S preloads the whole bank at EL2, and that is not optional. ECC is enabled
on this part, so a TCM location must be written before it can be read (TRM
6.2.2), and a stack is read before the program writes it -- the first context
restore pops what tx_thread_create built into it. The preload has to happen
before any C runs, because the demo images do not run bsp_boot.c, which is where
the ATCM preload lives. 32-bit stores suffice for BTCM where ATCM needs 64-bit.
Why BTCM for a stack: 16 KB at zero wait states where ATCM has one, and never
cached whatever the MPU says about it. Measured in the previous commit, uncached
BTCM comes within 6.6% of the best cached case while uncached DRAM0 is 22% off
it, so stack access runs at close to cache-hit speed without depending on a line
being resident. That is the property a determinism argument needs.
What this commit does not claim: no thread-level timing improvement has been
measured. The case for BTCM here rests on the memory characterisation and on
removing the cache from the path, not on a measured context-switch figure. That
measurement is worth doing and has not been done.
Verified on the S32Z280-594EVB. The demo reports its stack addresses so the
placement is visible rather than implied -- sleeper at 0x30100000 in BTCM,
spinner and judge in DRAM0 -- and passes with 100 ticks, 20 sleeper wakeups and
20 preemptions, so a real thread schedules, preempts and context-switches on a
tightly-coupled-memory stack. The boot image still passes six of six probes.
Assisted-by: Claude Code (Opus 5) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The TCM work so far enabled ATCM and left it empty, which buys nothing. This places code in it and measures the result.
Mechanism
link.ldsgains anATCMmemory region and an.atcm_textsection whose run address is in the bank and whose load address is inCODE.tcm_copy_atcm_textmoves it at boot.Two details are load-bearing rather than tidy:
IMP_MEMPROTCTLR = 0x11) and ATCM requires them where BTCM and CTCM accept 32-bit (Cortex-R52 TRM 6.2.2). Both ends of the section are 8-byte aligned so there is no narrower tail that would be left without valid check bits.0xA5A5A5A5.Measurement design
s32z280_atcm.elfis the same image ass32z280_boot.elfwith the interrupt service body in ATCM. Both targets exist so the comparison can be repeated on one board in one session without reconfiguring.The service routine is split into a timed wrapper that stays in
.textand a body that moves, so the wrapper's own cost appears in both measurements and cancels. Timing uses the PMU cycle counter, notCNTPCT: at 8 MHz the latter cannot resolve a handler body, let alone the variation in one.Results
64 samples each, caches enabled in both:
Reading the result honestly
The level shift is solid. ATCM is a quarter faster even though caches were enabled and code RAM had the instruction cache available. That says the handler does not stay resident between interrupts 10 ms apart, so each one pays a cold fetch from code RAM — which runs at half the core frequency, where ATCM runs at full speed with one wait state (S32Z2 RM 6.3.6). That is the realistic ISR case.
The determinism claim deserves less weight than the headline suggests. The spread narrows by only 16%, and ATCM's worst case (466) still sits slightly above code RAM's best case (454), so the distributions overlap at the tails rather than separating cleanly. Whatever jitter remains is not dominated by instruction fetch — GIC acknowledge, the timer re-arm and data accesses are all still in code RAM and DRAM. Anyone using this to support a determinism argument should know that.
Verification
Both images pass six of six boot probes. Full build clean, no warnings. The FVP example is untouched.