Repository navigation
uci-m3: unfence UCI register reads (11.6x faster large fetch); M3_READ_FENCE=1 restores the old client - #296
Merged
Merged
Conversation
The uci-m3 adapter fenced every UCI register access, reads included: two
uci_fence (~114 us each at 48 MHz) per body byte in net_poll's copy loop,
plus one after every status/data/status-line read in m3_cmd.s. The
1541ultimate firmware side reports that reads need no gap, only correct
state polling, so reads are now unfenced. Every WRITE ($DF1C control,
$DF1D command bytes) keeps its fence, and m3_begin/m3_exec keep the push
rule unchanged.
The copy loop also stops recomputing the SMC store address per byte: it
counts with X/Y and moves the store pointer, wrapping at the ring's end.
DATA_AV is still tested before every byte (ER-12), an over-long block is
still dropped and flagged, and Data More still ABORTs (ER-5).
Knobs (uci-m3 only, refused elsewhere, both in flags.stamp):
M3_READ_FENCE=1 every read fenced again; assembles the pre-change
client byte for byte (f5d9278b default target,
a3c67976 with the wikipedia REU-sink target).
M3_TRACE=1 per-net_poll timing records ($6000-$9FFF, link-
asserted free) for the per-READ round trip; never in
a default build.
tools/check_m3_client.py: test_read_pacing traces every UCI access in
the model and counts uci_fence executions after it (fence on every
write; none after any read, or after every read on an M3_READ_FENCE=1
image; <= 10 instructions per body byte), and test_ring_wraps_in_one_block.
tools/mutate_m3_client.py: nine mutants for the above, all killed.
tools/uci/rig_https_m3.py: a `timed` mode for the hardware A/B.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… screen The menu loop polls with no session, which filled the 1,365 records before 'G' on the first hardware run. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The parser report (the NewPP / transclusion / cache-key comments and the wgPageParseReport object) changes on every render; a difference inside it, or right after a per-render key, is metadata. Anything else, a short body, or a dropped block is DIFFERENT. TURBO_MHZ=0 divided by zero in rig_https_local's import, so 1 MHz is TURBO_OFF=1. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ced client Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e read budget Amendments from the adv-296 review: - test_copy_counts_at_page_edges: headers 1/255/256/257/512/768/893 at ring tails $0000/$0E7D/$0F00/$0FFF, plus short, exact-request and over-claim cases. The mutant count-always-iny (an unconditional iny at @p_copy_init) killed every session whose header is a multiple of 256 and no test fed one. - _pacing_flows traces a CLOSE whose PUSH is refused once, so the rejected path's CLR_ERR write fence is checked (rejected-clr-unfenced). - test_read_pacing's read side was SBC #1-shaped only; it now also caps the instructions after each read (READ_GAP_BUDGET, from the measured maxima), whatever the delay's shape (statline-nopsled). The docstring says what it detects and on which flows. check 53/53 (default 44648860, M3_READ_FENCE f5d9278b); mutants 74/74. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…he model adv-296 round 2: - body_oracle excused any changed token run that STARTED within 120 B after a per-render key, whatever its length (to end of line: up to 6,022 B on the wiki page). A run must now lie wholly inside a parser report span or the 120 B after a key: at most 3,175 B (0.42%). A bit flip 2,964 B past wgRequestId is now DIFFERENT; all 22 saved complete large bodies stay metadata-only. - check_m3_client refuses M3_TRACE=1 images (exit 2, like the mutator): m3_tr_end after net_poll's last read was a READ_GAP_BUDGET false alarm. - test_read_pacing's docstring names the budget's blind spot: a delay made of UCI register accesses restarts the gap. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e red cases adv-296 round 3: report_spans and the key windows came from the body under test, so a corruption could widen its own excuse. Breaking the wgPageParseReport object's closing } or a report comment's --> made the span run to the next --> or to EOF, and a 250 B block across the }, a block from there to EOF and a block across a --> were all accepted. The excusable regions now come from the REFERENCE. A region counts only if its delimiters (<!-- / -->, the object's key and }, a per-render key) are bytes the line+token diff found unchanged, which also places it in the body. A changed run is metadata only if its ref extent AND its body extent lie inside the same region. A key's window ends at its value (the first , } ] < or newline, at most 120 B). Bound on the wiki page: 3,106 of 751,822 B (0.41%). tools/check_m3_oracle.py: 11 synthetic cases (no fetched content). The old oracle (fccf639) fails 4 of them (1a, 1b, 1c, 2b); this one passes all 11. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
JC-000
force-pushed
the
perf/uci-m3-read-pacing
branch
from
October 14, 2026 05:11
408f657 to
5d32583
Compare
… rule adv-296 round 4: - A delimited region's body side could hold an insertion of any length that kept the line count (50,000 B inside the NewPP comment passed). Its body length may now differ from the reference's by at most REGION_SLACK = 64 B. Measured over the 33 complete saved bodies: every region was 0 or +1 B longer. - `-->` now ends a key's value window (a "Render ID ... -->" window ran to end of line). - check_m3_oracle.py gains 9 cases (7, 7b, 8-14: closer overwritten, 50 B in / 6 B out right after the object's }, "x,yy" at a value end, 50 KB inside a comment, a changed -->, <!-- and key) and a --mutants mode that breaks each rule in turn: closer/opener not checked, no body extent, no reference extent, flat 120 B windows, no --> end, no slack cap. 20/20 cases pass; 7/7 mutants killed. Bounds on the wiki page: 3,103 B (0.41%) of the reference; at most 3,103 + 32 regions x 64 = 5,151 B (0.69%) of the body. All 35 saved large bodies: 33 metadata-only, 2 DIFFERENT, as before. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
JC-000
added a commit
that referenced
this pull request
Oct 14, 2026
…ores them The plain UCI adapter put a uci_fence after every register access. Reads need none (the queues advance on the read, DATA_AV/STAT_AV are polled before every byte); measured on the U64E with the fenced 6510 comb build, read fences were 200 s of 363.6 s on the 751,823 B wiki page (#296's fairness measurement). Every WRITE fence stays, and so do the push rule, the shared uci_fence macro and every TOD-bounded wait. The 15 read sites in net.s (5) and uci_cmd.s (10) now use uci_read_fence (or an .ifdef around their jsr uci_settle). The trust store's uci_dos.s reads are left fenced: that path cannot be exercised on the C64U (no SD card), so it is not changed here. UCI_READ_FENCE=1 (BACKEND=uci only, in flags.stamp) assembles the previous images byte for byte: uci-onchip 375f9480, uci-comb 74a7d358, plain uci b4e4e6fb. uci-m3 (44648860) and ip65 are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
JC-000
added a commit
that referenced
this pull request
Oct 15, 2026
…t); measured-image provenance adv-296 on #300: #952 is the proposal issue and links no draft PR, no upstream PR implements $21/$23/$25 yet (#953 is GET_RANDOM only), and esp-tls/m3-master is not public. The default upstream status now says so, UPSTREAM_PRS still overrides it, and UPSTREAM_RUN no longer presupposes an upstream build. The NOT-measured branch says the measured PRG is reproducible from 71b047e (the merge of #296) and was built from 9eabd50 on 2026-10-13, and that with the adapter unchanged the PRG differs only by changes outside it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
JC-000
added a commit
that referenced
this pull request
Oct 15, 2026
…te over its include closure adv-296 round 2 on #300: - The source for $20-$25 IS public: JC-000/1541ultimate esp32-tls-offload (tip 72f33796, network_target.h defines $20-$25, the 90-94 lines are in network_target.cc), linked from GideonZ/1541ultimate#952. FIRMWARE_SOURCE says so, that the demo has not run on a build of it, and the upstream status says not upstream (GideonZ/1541ultimate#953 is GET_RANDOM only). - 'No demo image has been run on any firmware but the build named above' was false: the script's own history validated earlier images on esp-tls/m3 f2e46946. The line is now per-image (none recorded for a non-measured PRG) plus EARLIER_RUNS. - The adapter gate diffed src/net/uci-m3 only, so a change to src/net/uci/uci_regs.inc (uci_fence) moved the PRG and the README still said 'unchanged'. It now diffs every file in build/net/uci-m3/*.d plus cfg/c64-https-uci-m3.cfg, and names the files that changed. - #296, not #296. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The
BACKEND=uci-m3adapter put auci_fenceafter every UCI register access, reads included. That meant two fences (~114 µs each at 48 MHz) per body byte innet_poll's copy loop, plus one after every status, data and status-line read inm3_cmd.s. This PR removes the fences after reads of$DF1C-$DF1F, in the uci-m3 adapter only. The 1541ultimate firmware side says back-to-back reads need no gap, only correct state polling.Unchanged:
$DF1Ccontrol,$DF1Dcommand bytes) keeps its fence.m3_beginandm3_execkeep the push rule exactly as it was.src/net/uci/or the shareduci_fencemacro is touched.The copy loop also stops recomputing the SMC store address on every byte. It now counts with X/Y and advances the store pointer, wrapping at the end of the ring. It is 8 instructions per byte. Correctness is kept:
Knobs
Both are uci-m3 only, refused on other backends, and recorded in
flags.stamp:M3_READ_FENCE=1fences every read again. It assembles the pre-change client byte for byte:f5d9278b…for the default target, anda3c67976…withHTTPS_BODY_TO_REU=1 HTTPS_HOST=en.wikipedia.org HTTPS_PATH=/wiki/Commodore_64. The second one is the exact PRG the 2026-10-13 baseline A/B measured. It is the A/B arm and the fallback.M3_TRACE=1adds per-net_polltiming records (CIA2 timer B:A) at$6000-$9FFF, with a link-time assert that the image ends below that. It is for measurement only and is never in a default build. The default demo build's-Dset is unchanged, sobuild_m3_demo.sh's stamp check still accepts it (make package-m3-demoran green).Other profiles: byte-identical (sha256, clean builds before = after)
uci-m3 default:
f5d9278b…→44648860…. uci-m3 wikipedia target:e4814599…→24a81766…(this is the demo PRG).Model + mutation (red/green)
New tests in
tools/check_m3_client.py:test_read_pacingtraces every UCI access in the model on four flows only: a whole fetch, a CLOSE whose PUSH is refused once, a block longer than its header, and a Data More block. On those flows it fails if:SBC #1opcode fetches until the next access, against 0.9×OUTER×INNER.SBC #1runs after a read, or more instructions run after a read thanREAD_GAP_BUDGETallows ($DF1C 64, $DF1D 16, $DF1E 40, $DF1F 16; the measured maxima are 49/10/35/7). That catches a delay of any shape above those budgets, except a delay made of UCI register accesses (e.g. a loop of status reads): each access restarts the gap. A delay that fits under the budgets is not caught either; that is the detector's resolution.M3_READ_FENCE=1image, a read is unfenced.test_ring_wraps_in_one_block: one 893 B block across$CFFF→$C000, with nothing stored at$D000+.test_copy_counts_at_page_edges(added after review): headers 1/255/256/257/512/768/893 at ring tails $0000/$0E7D/$0F00/$0FFF; in each case every byte must arrive in order, the tail must be right, and nothing may be stored past the copy or at $D000+. Also covers short, exact-request and over-claim cases at 1–768.Results at the current head:
44648860): 53/53 passed.M3_READ_FENCE=1image (f5d9278b): 53/53 passed. On that image the same test asserts that every read is fenced.3b3685c9, anda3c67976withM3_READ_FENCE=1): 54/54 passed.24a81766): 53/53 passed.M3_TRACE=1images are now refused with exit 2 (e.g.531d3a1f). That rule came from review. On those imagesm3_tr_endruns after net_poll's last register access, andREAD_GAP_BUDGETcounted it as a delay after a read, which was a false alarm.M3_ALLOW_TLS12=1+HTTPS_PIN_SPKI_SHA256image fails the same 6 model tests (test_open_layout,test_non_tls13_session_is_refused,test_alert14_reads_as_no_tls13,test_hint_texts_reach_the_user,test_reply_version_major_checked,test_only_version_alerts_read_as_no_tls13) at this head, at 9eabd50 and at origin/master 75fe1b4. Those tests assume the default TLS-1.3-only image.tools/mutate_m3_client.py: 74 mutants, 74 killed: the 9 below plus 3 from review. The 9 new mutants and what they produced (RED on each injected regression):The review (adv-296) found three survivors. Each is now RED on the mutant and GREEN on this PR (model check 53/53 on
44648860and onf5d9278b):count-always-inywas a real test gap. An unconditionalinyat@p_copy_initwould kill every session whose READ header is a multiple of 256, and no test fed such a header. The code in this PR does not have that defect.The runner now refuses an
M3_READ_FENCE/M3_TRACEbuild, because the pacing mutants target the default image.Hardware A/B (U64E)
Device: U64E,
GET /v1/info= fw 3.15 git3a1ff9ff, FPGA 126, core 1.50, read under the DeviceLock. 48 MHz, REU Enabled 16 MB.Rig:
tools/uci/rig_https_m3.py timed(new mode):$10:0000), 'Q', then the body is read back from the REU.M3_READ_FENCE=1(a3c67976), S = this PR (3b3685c9). Both useHTTPS_BODY_TO_REU=1 HTTPS_HOST=en.wikipedia.org HTTPS_PATH=/wiki/Commodore_64.net_last_error $00./robots.txt28,275 B/robots.txt28,275 B/wiki/Commodore_64751,823 B/wiki/Commodore_64751,823 BOn the large page that is 11.6× faster. Bulk transfer after the handshake is ≈13.4 s, ≈56 KB/s. Timing resolution is about ±0.6 s (0.5 s screen poll plus REST latency).
Independent runs by the adversary (adv-296): U64E fw
3a1ff9ff, PRG3b3685c9, 48 MHz, large page 17.87 / 20.07 / 18.36 / 18.11 s, all PASS. At 1 MHz, large page: 281.23 s, PASS, CRC9C298DA4. That closes the "large page not timed at 1 MHz" gap below.Body oracles (each run compared against a host
curlmade just before it, with the client's UA):/robots.txt: all 6 runs are byte-equal to the reference (CRC-3279C6EE37).report_spans(a947c9b) was added afterwards.wgRequestIdwas accepted as metadata-only.}or a comment's-->stretched the span to the next-->or to EOF, and blocks across them were accepted.wgPageParseReportobject, and the value after each per-render key (up to the first, } ] <, newline or-->, at most 120 B);REGION_SLACK= 64 B. Before this cap, a 50,000 B insertion inside the NewPP comment was accepted. Across the 33 complete saved bodies, every region was 0 or +1 B longer in the body. 64 B allows for re-rendered numbers, timestamps, ids and host names changing length.}, from there to EOF, and across a-->, plus the bit flip 2,964 B pastwgRequestId, are all now DIFFERENT. The controls (tail-only, gap-only) remain DIFFERENT.tools/check_m3_oracle.py(new, standalone) pins this with 20 synthetic cases, so no fetched content is committed.--mutantsmode breaks each rule in turn. 7/7 killed:-->end (12)wgRequestIdvalue. The length is pinned separately, by the framing.20261013_205230: 8 metadata-only.20261013_215428: 9 metadata-only, 1 DIFFERENT (the early-viewer F attempt, see below).20261013_221522: 6 metadata-only.20261013_230122: 5 metadata-only, 1 DIFFERENT (its host reference was an experiment render; the C64 body has the usual CRC).20261013_214256: 4 metadata-only.20261013_214652: 1 metadata-only.221522.9C298DA4/ 751,823 B, on every arm (fenced, this PR, uci-m3 stage 2: READ 1472 B over a Data More continuation (measured: no gain; draft) #297, the trace builds, and the 6510 builds of the fairness runs). The adversary's independent runs and the baseline session's six have the same CRC.20261013_215428, attempt 22:04:55, PRGa3c67976(M3_READ_FENCE=1, byte-identical to master's client), the REU viewer opened at 360,448 of 751,823 B: HTTP 200,net_last_error $00, CRCB191A9F3.wgTestKitchenUserExperiments), and it ends mid-tag. 360,448 is 0x58000.m3_eof_code, so it cannot be attributed to the server (a legitimately terminated response) or to the client./wiki/Commodore_64: all 6 runs are byte-identical to each other, CRC-329C298DA4. That is the same CRC the 2026-10-13 baseline session got on all six of its bodies. Against the fresh host fetch they differ only inside MediaWiki's per-render parser report. The oracle diffs lines, then tokens within changed lines, and accepts a difference only inside the parser-report comments or thewgPageParseReportobject, or right after a per-render key. Offline, the same oracle rejects one flipped byte, a 5,000 B truncation, and one dropped 893 B block.1 MHz (turbo off), S,
/robots.txt, n=1: 16.22 s, byte-equal. The baseline was 373.5 s (n=1). A second 1 MHz run on the trace build took 16.38 s.Demo disk (
make package-m3-demo, PRG24a81766…,rig_https_m3_demo.py), n=1:/wiki/Commodore_64: 751,823 B == Content-Length, 18 s./robots.txt: 4 s.94.The demo README figures (
build_m3_demo.sh) are updated from this run.Per-READ round trip (M3_TRACE, for the firmware team)
The round trip is measured from the PUSH of a READ until REPLY_VALID is seen. It includes the ~0.114 ms post-PUSH write fence, which sets the floor. Ticks are CIA2 PHI2 cycles at 1.0227 MHz. Build ST =
dd82a93e…(fast + trace), 48 MHz,/wiki/Commodore_64. It made 1,666 READs while connected; the first 1,365 were recorded.$FFFF+02,NO DATA: 11)/robots.txtat 48 MHz: 893 B reads had a median RTT of 5.17 ms (n=30), small reads 0.17 ms (n=32).$DF1Cbit 3 (error_busy, a refused PUSH) was read after every PUSH and its fence, on all four trace runs: 87 / 1,687 / 85 / 1,685 pushes (the fenced trace build isc67447c0…), plus 85 at 1 MHz. It was never set: 0 in every run, and the post-PUSH values seen were only$10,$11and$E0.The push rule is unchanged:
m3_beginwaits for state 00 with abort-pending and busy clear.m3_execclears bit 3, PUSHes, then waits for reply-or-error.M3_EXEC_REJECTED, not retried. No refusal was observed, so no retry was added.Not done / caveats
Follow-up: 1472 B READs (measured null result, #297)
3a1ff9ff:NET_MAX_SOCKET_READ1472,start_read_reply/get_more_data.NET_TLS_PT_MAX=TLS_STEP_OUT_MAX= 1024. The 1541ultimate side confirmed this from source, bounded underneath by the UART RPCCMD_BUF_SIZE.3b3685c9).🤖 Generated with Claude Code