Measured context-switch cost against the memory holding the stack - #635
Merged
fdesbiens merged 1 commit intoAug 17, 2026
Merged
Conversation
eclipse-threadx#634 placed a thread stack in BTCM and deliberately claimed no timing benefit, because none had been measured. This measures it. Two pairs of equal-priority threads hand control back and forth with tx_thread_relinquish. One pair has both stacks in BTCM, the other in DRAM0, and the measuring thread of each pair times the round trip in PMU cycles. Both pairs run in one image from one copy of the measuring code, which is what makes the comparison safe: the alignment trap that invalidated earlier work here bites when two builds with different layouts are compared, and a code shift moves both pairs equally. Reproducible to the cycle across runs: min mean max BTCM stacks 1370 1379 1402 DRAM0 stacks 1476 1480 1508 BTCM is about 7.4% faster, or 110 cycles on a round trip of two switches. Three findings that bound the claim, and the last one deflates it. The figure holds whether the cache is warm or cold. Cleaning and invalidating the data cache before every timed switch costs both configurations about 40 cycles and leaves the gap at 7.4%: warm it is 1334 against 1440, cold 1370 against 1476. So the advantage comes from BTCM's zero wait states, not from avoiding cache misses. That is because a context switch touches almost no stack -- sixteen registers, about one cache line -- so the stack's cache state has little to contribute either way. TCM should matter much more for threads with deep call chains or large locals, where the stack working set is big enough for cache state to dominate. That is not measured here and should not be assumed. There is no determinism benefit visible in this test. Excluding preempted samples, jitter is 32 cycles for BTCM and 28 to 34 for DRAM0 -- comparable, not better. A first version of this measurement appeared to show BTCM with 16 times less jitter, and that was wrong: max was reporting whichever pair a timer tick had landed on. Across three runs the outlier appeared in the BTCM pair once and the DRAM0 pair twice. Samples past 2000 cycles are now counted separately and excluded from min, mean and max alike, and the count is printed so the reader can see how many there were. Also tried and discarded: loading the partner thread with a cache walk to create pressure. The timed round trip includes the partner, so the walk dominated every sample and put all 256 past the outlier threshold. The per-sample flush replaced it and sits outside the timestamps. The demo also starts the PMU cycle counter, which bsp_boot.c does for the probe image and this image never ran. Without it every reading would have been zero, which reads as a free context switch rather than as a dead counter. Assisted-by: Claude Code (Opus 5) <noreply@anthropic.com>
fdesbiens
added a commit
that referenced
this pull request
Aug 17, 2026
#635 found BTCM worth about 7.4% on a context switch with no determinism advantage, and said why: a switch saves sixteen registers, roughly one cache line, so the stack's cache state has almost nothing to contribute. It named the interesting case as work with a large stack working set, said it had not been measured, and said it should not be assumed. This measures it, and the answer inverts the earlier one. deep_touch recurses 24 frames, writing a frame on the way down and reading it on the way up, so the working set is the whole descent. The cache is cleaned and invalidated before each sample, so every descent starts cold. Two threads, one stack in BTCM and one in DRAM0, no partner threads and no relinquish: the timed region is entirely within one thread. Reproducible across runs: min mean max spread stack in BTCM 42868 43031 44890 2020 stack in DRAM0 42982 43291 49842 6860 The mean is the same to within 0.6%. The worst case is 10% lower for BTCM and the spread is 3.4 times tighter. No sample in either configuration exceeded twice the minimum, so these maxima are the workload rather than a timer tick -- which is the mistake that produced a false jitter result in #635 and is why the count of interrupted samples is printed. Put beside #635 the two measurements say opposite things and both are true. For a context switch, a small footprint touched every time, BTCM buys throughput and no determinism. For stack-heavy work, a large footprint touched once, it buys determinism and almost no throughput. The reason is that this workload is compute bound at the optimisation level this BSP builds at: 43000 cycles for 24 frames is dominated by call and loop overhead, so line fills are a few percent of the total and barely move the mean. What they do is vary, and that variance is what a bank with no cache in the path removes. So the determinism argument for TCM holds here, but it is worth 10% of worst case and a threefold narrowing of spread, not an order of magnitude. Anyone citing this in a safety argument should cite those numbers and not a larger claim. Assisted-by: Claude Code (Opus 5) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#634 placed a thread stack in BTCM and deliberately claimed no timing benefit, because none had been measured. This measures it.
Two pairs of equal-priority threads hand control back and forth with
tx_thread_relinquish. One pair has both stacks in BTCM, the other in DRAM0, and the measuring thread of each pair times the round trip in PMU cycles.Both pairs run in one image from one copy of the measuring code, which is what makes the comparison safe — the alignment trap that invalidated earlier work here (#631, #633) bites when two builds with different layouts are compared, and a code shift moves both pairs equally.
Result
Reproducible to the cycle across runs:
BTCM is about 7.4% faster — 110 cycles on a round trip of two switches.
Three findings that bound the claim
The figure holds warm or cold. Cleaning and invalidating the data cache before every timed switch costs both configurations ~40 cycles and leaves the gap at 7.4% (warm: 1334 vs 1440; cold: 1370 vs 1476). So the advantage comes from BTCM's zero wait states, not from avoiding cache misses.
That is because a context switch touches almost no stack — sixteen registers, about one cache line — so the stack's cache state has little to contribute either way. TCM should matter considerably more for threads with deep call chains or large locals, where the stack working set is big enough for cache state to dominate. That is not measured here and should not be assumed.
There is no determinism benefit visible in this test. Excluding preempted samples, jitter is 32 cycles for BTCM and 28–34 for DRAM0 — comparable, not better.
A wrong result, caught
A first version of this measurement appeared to show BTCM with 16× less jitter, and it was wrong.
maxwas reporting whichever pair a timer tick had landed on: across three runs the outlier appeared in the BTCM pair once and the DRAM0 pair twice, while min and mean held to the cycle. Samples past 2000 cycles are now counted separately and excluded from min, mean and max alike, and the count is printed so a reader can see how many there were.Also tried and discarded: loading the partner thread with a cache walk to create pressure. The timed round trip includes the partner, so the walk dominated every sample and put all 256 past the outlier threshold. The per-sample flush replaced it and sits outside the timestamps.
Incidental fix
The demo now starts the PMU cycle counter.
bsp_boot.cdoes this for the probe image and the demo never ran it, so every reading would have been zero — which reads as a free context switch rather than a dead counter.