Skip to content

PQ P4: hybrid ServerHello reader, KeyGen in tls_connect, Decaps after SH - #314

Merged
JC-000 merged 4 commits into
masterfrom
feat/pqc-p4-hybrid-sh
Oct 17, 2026
Merged

JC-000 merged 4 commits into
masterfrom
feat/pqc-p4-hybrid-sh

Conversation

@JC-000

@JC-000 JC-000 commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Scope (PQC phase P4, everything under PQ_HYBRID=1)

  1. tls_connect draws a 64 B d||z seed from the DRBG, runs ML-KEM KeyGen, zeroes the 2,400 B dk at once, and arms the sectioned ClientHello with the ek (11EC = ek || x25519, then 001D also offered, the shape PQ P3: sectioned ClientHello (16-bit lengths) + HelloRetryRequest refusal #312 sent to OpenSSL). The "arm before jsr tls_connect" caller contract is retired.
  2. A streaming ServerHello reader (src/crypto/pq_hybrid.s): any record fragmentation, a 16-bit field/message counter, nothing buffered, every record folded into the transcript exactly once and in order. 11EC needs exactly 1,120 B = ct 1,088 (into cert_buf) || x25519 32; 001D needs 32. Refusals have their own tls_sh_err codes $01-$0E (src/constants.inc).
  3. pq_sh_decaps runs after the drain and BEFORE the X25519 mult; tls_kex_group ($1D/$EC) and tls_mlkem_ss are recorded for P5.
  4. The key schedule is NOT hybrid (P5). Against an 11EC server the handshake therefore fails at the first encrypted record (documented known state). Against a 001D server the PQ image completes HTTPS as before.

Round 1 (adversary findings, all fixed in 71f4bfe)

# change red case / killer
1 oracle gate used tls_state < 3, never true for a failed run ($FF); now gates on tls_last_state when tls_state == $FF new unit cases state=ff,last=02 / 01 -> inconclusive (exit 78) in pq_keylog_check.py --selftest and test_pq_keylog_check.py
2 openssl_listener.py reported LISTENING after probing a port a foreign process held; now only when s_server prints ACCEPT, non-zero exit on bind failure/early death tools/test_openssl_listener.py (pytest): held port must exit non-zero. Red shown: the old launcher fails it (1/2), the new one passes (2/2)
3 "next prep scrubs ss" was untested glue test writes ss/seed, runs prep; mutant M45 (delete jsr pq_secrets_zero) killed
4 SH failed-poll budget re-armed per record; now ONE budget for the whole message test_pq_kem.py timeout: budget nearly spent after record 1, the next failures must end it (poll count 2..16). M46 (re-add the reset) killed (it wraps to 1)
5 unsolicited SH extensions refused (SH_ERR_EXT $0D); SS_SKIP/h_skip deleted pre_shared_key, cookie, unknown, empty, renegotiation_info, server_name, each before/between/after, plus after a fragmented hybrid key_share; M47 killed. Hardware: OpenSSL sends only supported_versions + key_share in both SH shapes (reruns below pass)
6 vacuous last_state == 3 or carry == 1 now and
7 001D key length high byte unpinned 288 B case; M55 killed
8 pq_sh_init's reset of tls_kex_group unpinned refusal after a success leaves group 0; M48 killed
9 PR body counts labelled below
10 new suites on a non-PQ build/ crashed with KeyError require_pq_image() -> cannot_run (exit 2, CANNOT RUN). Shown: master's PQ image + the new suites with C64_SKIP_BUILD=1 -> rc 2. A behavioural red against master's single-record parser is not meaningful: it cannot carry any of these fixtures (the 1,178 B SH does not fit its buffer), so the mutant table is the primary red
11 legacy_version no longer checked (RFC 8446 4.2.1); VER state discards its 2 bytes accepted-cases for 0302/0304/0000/0301
12 pq_mlkem_scrub (next to pq_mlkem_call, sei + $01=$34): zeroes $D000-$DFFF (s_hat / y,e1,e2) and the whole $B100-$BBFF window (m', K', r', Keccak state, X25519_TABLES, mul_dma) after KeyGen and after Decaps, on every outcome. 6,912 stores, about 76K cycles (11 cycles/store, estimated from the loop, not measured on a clock) direct wrapper test (fills both areas, bytes either side untouched, $01 restored); prep and Decaps tests read both areas; M49-M52 killed
13 single-record oversize SH: fixed (cheap): a malformed/oversize header while waiting for the SH is fatal in tls_rec_frame_fail (PQ only) and tls_recv_server_hello fails at once with SH_ERR_RECORD $0E test_pq_kem.py oversize (ERROR-latched, code $0E, < 8 polls); M53, M54, M56 killed. It still cannot READ such a hello
14 CLAUDE.md Memory layout lists the seed/ss in NET_BSS_TAIL

Round 2 (tests + docs only; no .s/.cfg/Makefile byte change)

  • pq_mlkem_scrub's $01 and I-flag restore are now asserted: directly (test_pq_sh.py 5, entered with I=1 then I=0), after pq_connect_prep and after pq_sh_decaps (test_pq_kem.py, first call entered with I=1, the second with I=0). Entry order matters: a leaked $01=$34 with I clear livelocks on the next IRQ (the RTI vector acknowledges nothing), so $01 is checked under sei first. New mutants, each killed BY ASSERTION in seconds, not by a hang: Y3 (no $01 restore) 4 s, Y4 (no php/plp) 4 s, Y3p (leak after KeyGen) 6 s, Y4d (I flag leaks after Decaps) 8 s. Y3 and Y4 first died only by timeout against the round-1 tests (no FAIL line); that is what this fixes. Green: test_pq_sh.py 5 8/8, test_pq_kem.py prep decaps 26/26.
  • Wording: mul_dma is per-row scratch regenerated before each row reads it (stale entries are never read); X25519_TABLES is rebuilt by x25519_*_fresh (engineering-notes, comment in mlkem_link.s).
  • Hashes unchanged (rebuilt after the comment edit): uci-onchip-pq 208f3967..., uci-comb-pq e97ab54a..., uci-onchip-store-pq eeeb8b0f..., uci-comb-store-pq 0efe0803...; knob-off uci-onchip 9e11f570... and uci-comb-store fee4d5b1... as in the table; hardware SNI onchip image still 493cee70bb40957bb9ba8172d6dc91ab5b2cb4b1ec917b8919f682ba32751ceb.

Notes for P5

Decisions (and why)

  • Both shares offered; matches PQ P3: sectioned ClientHello (16-bit lengths) + HelloRetryRequest refusal #312's interop.
  • Homes: ek AND ct share cert_buf (ek dead once the CH is sent; ct dead once Decaps returns; Decaps precedes anything that can write cert_buf; the comb cold bank refuses to run during a session). Seed (64) + ss (32) are one adjacent pair in NET_BSS_TAIL (below $4200), asserted at link, one 96 B session_scrub span.
  • Code size ~930 B (first estimate 300 B): tls_parse_server_hello (~280 B of LOADER) is assembled out under PQ; two optional segments PQ_CODE / PQ_GLUE, LOADER/NET_CODE on cfg/c64-https-uci.cfg, CRYPTO_HOT on both comb cfgs.
  • TLS_HS_SEQ_CHECK not reused: the header state (type 2), the length bound and DONE give the same guarantee for the first message.
  • Stricter than the old parser (PQ image only): empty session-id echo, extension block fills the message exactly, supported_versions and key_share exactly once, 11EC only if offered, every other extension refused.
  • Secrets: dk zeroed straight after KeyGen even if refused; seed zeroed after Decaps and by tls_connect's error exit; ML-KEM residue scrubbed as above; tls_mlkem_ss is KEPT on error like tls_shared_secret (the hardware oracle reads it; 'Q' or the next prep scrubs it).

Memory (tools/measure_margins.py --build, free bytes, v1 -> now)

profile LOADER NET_CODE CRYPTO_OVERLAY CRYPTO_HOT NET_BSS_TAIL
uci-onchip-pq 129 -> 179 222 -> 166 2,578 509 1,263
uci-comb-pq 459 -> 457 381 -> 335 325 463 -> 505 1,263
uci-onchip-store-pq 107 -> 157 222 -> 166 372 196 1,263
uci-comb-store-pq (tightest) 344 -> 342 243 -> 197 55 335 -> 377 1,263

Knob-off PRGs byte-identical to master 10ccf25 (sha256, before == after, re-verified after round 1)

profile sha256
ip65 3fccbeb13d343b55d5a9a4fc7a59e9a4a09b9a3aaee26b65c497bc8f34a4354e
ip65-onchip e69bfce7b552e765df02cb253b4ff12cecaca3e978669376b7af5ffad9343dce
uci fe3b6c204d95fcdb3e818fcdc43275b72bd8ec5adc14fb28f7dfe26902846d20
uci-onchip 9e11f570f219e7fcb6bd9ba0f8398ce8fee28f930d8022941c74d6c7f1f1f9e6
uci-comb ad85797b67320b81441d6a651c12c164465e642c26fea346fc4e3d55a13e4a14
uci-onchip-store 1e82434885d87aa4e7a34d57230dc805759ddf18d3a081d95a5c7e85d353e70b
uci-comb-store fee4d5b107b2f848bdaa41b6855aef3f2a2032920cf5a893fe8c71b2fa3e2f2b

(also identical: ip65-wiki, ip65-onchip-wiki, uci-comb-demo-store). PQ images (non-SNI): uci-onchip-pq 208f3967..., uci-comb-pq e97ab54a..., uci-onchip-store-pq eeeb8b0f..., uci-comb-store-pq 0efe0803....

Tests (exact command for every count; all on this head)

  • python3 tools/test_pq_sh.py (builds the PQ uci-onchip image): 186/186.
  • python3 tools/test_pq_kem.py: 71/71 (KeyGen/Decaps vs the libs/mlkem reference, CH wire, tls_connect end to end with breakpoints, refused SH, refused KeyGen, oversize record, shared timeout budget, residue zero).
  • python3 tools/test_pq_keylog_check.py and python3 tools/pq_keylog_check.py --selftest; python3 tools/test_openssl_listener.py (2/2).
  • python3 tools/test_mlkem_placement.py: 194/194. python3 tools/test_mlkem_linked.py --no-mutants: 25/25; with mutants (python3 tools/test_mlkem_linked.py): 34/34.
  • BACKEND=uci python3 tools/test_hs_sequence.py: 62/62. BACKEND=uci python3 tools/test_tls_handshake.py: 36/36. test_prg_load.py 13/13, test_runner_coverage.py, test_pytest_boundary.py, test_rig_scratch.py, test_build_flags_stamp.py 21/21 pass. Bare python3 -m pytest -q: 507 passed.
  • Red: the new suites on a non-PQ build exit 2 (CANNOT RUN); against the mutants below they go red.

Mutants (python3 tools/mutate_pq_sh.py, 56, all killed, green controls first)

Round 0: 44 mutants (39 killed first pass, 5 survivors strengthened and re-killed). Round 1 adds M45-M56 (table above), plus re-targeted M20 for the scrub call now in the KeyGen path. Full table: python3 tools/mutate_pq_sh.py --list (mutant, file, edit, suite). One mutant (M17) removes bcs and cmp #0 together on purpose: either alone is equivalent for a wrapper refusal (A=$FF).

Hardware rerun after the parser changes (U64E 10.43.23.81, DeviceLock, OpenSSL 3.6.5 s_server, free ports 24510-24513, port 4433 untouched)

  • onchip: make BACKEND=uci USE_NISTCURVES_ONCHIP=1 PQ_HYBRID=1 HTTPS_SNI=www.foo.invalid, sha256 493cee70bb40957bb9ba8172d6dc91ab5b2cb4b1ec917b8919f682ba32751ceb, 48 MHz. -groups X25519: full HTTPS PASS (95 s). -groups X25519MLKEM768: SH accepted (kex $EC), Decaps ran, expected failure at the first encrypted record; oracle (pq_keylog_check.py --state) exit 0 MATCH.
  • comb: make BACKEND=uci USE_NISTCURVES_ONCHIP_COMB=1 PQ_HYBRID=1 HTTPS_SNI=www.foo.invalid, sha256 d9fe6881fbc905221b645e73b7e9a79323e09ca639839f2901705d860397e73b, 48 MHz, REU 16 MB. X25519: PASS (191 s). Hybrid: expected failure, oracle MATCH.
  • (Round 0 images 8234c608... / e5ff2aa2... passed the same runs; boot_check passed on 8234c608....)

Known gaps

  • Key schedule not hybrid (P5); 11EC servers fail at the first encrypted record by design.
  • A server that ignores max_fragment_length and sends the 1,178 B SH as ONE record now fails at once (SH_ERR_RECORD) but cannot be read (tls_rec_buf is 548 B).
  • CCS records between SH fragments are skipped by the record layer (as before).
  • tls_mlkem_ss survives a failed connect (deliberate).
  • Scrub cycle count is an estimate from the loop, not a clock measurement.

🤖 Generated with Claude Code

JC-000 and others added 4 commits October 17, 2026 10:03
…ter SH (WIP)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Streaming ServerHello machine (any record fragmentation), KeyGen + hybrid
key share armed inside tls_connect, Decaps before the X25519 mult, seed/ss
homes with a scrub span, red/green suites, 44-mutant harness, hardware
oracle against openssl s_server -keylogfile.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…, one SH timeout budget, fail fast on oversize SH record, oracle/listener/test fixes

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…Y4 mutants), mul_dma wording; tests and docs only

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@JC-000
JC-000 marked this pull request as ready for review October 17, 2026 18:17
@JC-000
JC-000 merged commit 4e45a9b into master Oct 17, 2026
@JC-000
JC-000 deleted the feat/pqc-p4-hybrid-sh branch October 17, 2026 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant