Repository navigation
Conversation
Contributor
|
fughilli
force-pushed
the
diffuse_capture
branch
from
September 7, 2026 00:37
0ae8e3f to
b53363b
Compare
fughilli
pushed a commit
that referenced
this pull request
Sep 21, 2026
…152 decoder infra) The mapping journey needs a camera; under the app-driver (?driver=) there is none. SyntheticCaptureSource renders a KNOWN fixture's LEDs — projected through a pinhole camera on an arc (geom/pinhole.project) and coloured with the SAME hue-code the firmware drives (code/gray.colorForFrame) at the SAME frame timing (code/timing.frameIndexAt) — into a pre-reduced RGBA frame (alpha = lit mask, RGB = colour), exactly the "virtual image → decoder" convention the synthetic pipeline test is built on (web/tests/pipeline_synthetic.test.ts). So the real detector CCLs the blobs and the real decoder recovers LED indices, with no camera and no scene. capture.ts takes a driverActive() branch (mirrors the iOS native reduced-frame path: offscreen GL context, no-op exposure target) and hands the source the negotiated code-book (CodeParams + epoch + clock) right after startMapping resolves, so the render matches what the decoder expects. Verified: //web:web_ts_typecheck_test + //web:web_tests_js compile. Loaded only behind the guard → zero production bundle impact. Next: a device backend (mock WSS server or the rig C6) to run the browser smoke end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fughilli
added a commit
that referenced
this pull request
Sep 21, 2026
… camera/IMU + Android Nix lane) (#177) * pi/hitl/phone,web: app-driver WebSocket control channel + virtual BLE (phone HITL MVP) Phase 0 of the phone-in-the-loop HITL station: drive the web/PWA app through the basic user journeys (scan/connect, mapping, gamma/hardware config) over a WebSocket, so we can debug the pairing/connection/mapping flow with a phone (emulator now, real device + gantry later) in the loop. Web app (guarded by ?driver=<ws-url>, dynamic-imported → zero production cost, mirrors the shipped ?demo= seam): - web/src/driver/{guard,harness}.ts — the app-driver: opens a WS to the station, streams appState transitions + milestones, and dispatches JSON commands to the exact production functions a user's taps invoke (appState.connect, provisionViaBle, client.startMapping/stopMapping/setHardwareConfig/setColorCorrection, router.navigate). - web/src/net/virtualBle.ts — virtual Improv + player BLE devices (promoted from the in-line test fakes), swapped in behind the guard at requestImprovDevice()/ requestBleDevice() so journeys run headless with no radio. - ui/app/main.ts — the ?driver= branch, installs the guard before any screen mounts. Station (pi/hitl/phone/, uses websockets like the rest of the harness): - driver_server.py — the WS server + async driving API (command/query/wait_event). - journeys.py — the three journeys asserting on the app's real replies. - launcher.py — headless-Chromium (serves //web:dist) + Android-emulator lanes. - phone_e2e.py + BUILD.bazel — the runner. Verified: //web:web_ts_typecheck_test + the BLE unit tests pass; //pi/hitl/phone:phone builds. Next: synthetic camera source (mapping journey) + a browser smoke run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * web: synthetic capture source for the phone HITL harness (reuses the #152 decoder infra) The mapping journey needs a camera; under the app-driver (?driver=) there is none. SyntheticCaptureSource renders a KNOWN fixture's LEDs — projected through a pinhole camera on an arc (geom/pinhole.project) and coloured with the SAME hue-code the firmware drives (code/gray.colorForFrame) at the SAME frame timing (code/timing.frameIndexAt) — into a pre-reduced RGBA frame (alpha = lit mask, RGB = colour), exactly the "virtual image → decoder" convention the synthetic pipeline test is built on (web/tests/pipeline_synthetic.test.ts). So the real detector CCLs the blobs and the real decoder recovers LED indices, with no camera and no scene. capture.ts takes a driverActive() branch (mirrors the iOS native reduced-frame path: offscreen GL context, no-op exposure target) and hands the source the negotiated code-book (CodeParams + epoch + clock) right after startMapping resolves, so the render matches what the decoder expects. Verified: //web:web_ts_typecheck_test + //web:web_tests_js compile. Loaded only behind the guard → zero production bundle impact. Next: a device backend (mock WSS server or the rig C6) to run the browser smoke end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: runnable browser lane — headless Chromium drives the app-driver loop The station now brings the real web app up in headless Chromium (Playwright, async) serving //web:dist at ?driver=, waits for it to connect back, and runs journeys. A device-free `smoke` journey exercises the whole loop with NO backend: navigate + virtual-BLE provisionBle (the virtual Improv peripheral answers the RPC in-app), so it validates the app-driver + guard + virtual BLE end to end. VERIFIED end to end: `bazel run //pi/hitl/phone:phone_e2e -- --browser --journeys smoke` loads the app, the app connects back over the driver WS ("app ready"), and the station drives it to a provisioning redirect ("PASS smoke", "ALL JOURNEYS PASSED"). - launcher.py: ensure_chromium() (one-time Playwright install, like the docs capturer) + async open_chromium(); dropped the raw-subprocess launcher. - phone_e2e.py: Playwright browser lane + a journeys map runner. - journeys.py: journey_smoke (device-free). - BUILD: + playwright dep. - .claude-container-overlay: record the Chromium runtime libs (libnss3/libgbm1/… t64) so the browser lane is reproducible across container relaunches; + the apt-install helper it was recorded with. The connect/mapping/config journeys still need a device backend (--device-ws to a rig-forwarded C6 or a mock) — next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: mock device backend — connect + config journeys green in-container A self-contained mock splanc device (mock_device.py) speaking the ledmapper.v1 wire via the real Pi-server codec (pi/server proto_wire): hello->welcome, time_sync, get/set_hardware_config->hardware_config_state, set_color_correction->welcome, start/stop_mapping + uploads. It handles the firmware-only config messages the real Pi reconstruction server doesn't, so the config journey needs no ESP32-C6; point --device-ws at a rig C6 to exercise real firmware instead. phone_e2e gains --mock-device (spins the mock up in-process, points the app at it). VERIFIED end to end in headless Chromium: bazel run //pi/hitl/phone:phone_e2e -- --browser --mock-device --journeys connect,config → PASS connect (virtual BLE provision → connect → welcome MAC) → PASS config (setHardwareConfig → hardware_config_state; setColorCorrection → welcome) → ALL JOURNEYS PASSED journey_config drives the client config RPCs directly rather than mounting the Hardware Setup screen — that screen fires its own getHardwareConfig which would collide with setHardwareConfig on the same reply type (one pending request per reply). Remaining: the mapping journey (the capture-screen lifecycle drives start/stop itself + solve placement) is the harder follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: user journeys as declarative JSON data (inputs + expectations) Move the three user journeys from imperative Python to STRUCTURED DATA — Maestro-style (declarative flows + reusable subflows) but over our semantic app-driver protocol, not DOM selectors (surveyed Maestro / Chrome DevTools Recorder / Puppeteer Replay; all are DOM-oriented, so we borrow the shape, not the schema). - journeys/*.json — each journey is {name, description, inputs, steps}. A step is {"do"|"query": <method>, "with": {...}, "expect": {...}} or {"await": "connected"| "milestone:<name>"|"state:<s>"} or {"runFlow": "<journey>"} (reusable subflow). `with` values interpolate ${var} from inputs+runtime context (an exact "${var}" preserves the value's type). Expect DSL: present / nonempty / equals / contains, with dotted paths into nested replies. - journey_runner.py — loads the JSON, interpolates, dispatches to the app-driver, and asserts each step's expectations. Replaces the imperative journeys.py. - phone_e2e wires the registry + a runtime context (device_ws / ssid / led_count). VERIFIED: `--browser --mock-device --journeys smoke,config` → PASS (config runFlow's connect, so it's self-contained). The format maps 1:1 onto the driver, so new journeys are data, not code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: real rig-reservation device backend (journeys vs real firmware) Adds --reservation alongside --mock-device so the phone journeys run against a REAL ESP32-C6 on the HITL rig, not just the mock. reservation_backend.py reserves a C6, flashes the netstack player bundle, provisions it onto the rig AP over BLE Improv, and res.forward()s its wss:443 to localhost — reusing the existing harness (hitl_client. Reservation + provision.provision_dut/dut_target). It's a sync context manager entered before the async run; the tunnel + heartbeat live in their own subprocess/thread so they survive asyncio.run. VERIFIED end to end on hitl-rig-2 (real c6-003f08): reserve → flash → BLE-provision (DUT at 10.42.0.28) → forward → headless Chromium connects to the real C6 over wss → config journey PASSES against real firmware (welcome MAC, set_hardware_config -> hardware_config_state, set_color_correction -> welcome), then releases cleanly. So the same declarative journeys now run against either backend: --browser --mock-device --journeys smoke,config (self-contained, no rig) --browser --reservation --journeys config (real firmware via the rig) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: station README + MVP BOM Document the phone-in-the-loop HITL station in one place (until now it lived only in commit messages): quickstart for each lane/backend, how the app-driver + virtual BLE + synthetic camera fit together, the declarative journey-JSON format + app-driver method reference, and the MVP bill of materials (Android/Linux station first, iOS/Mac later, shared rig) with a forward-looking gantry/ambient-light note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: Android emulator lane — SDK + adb via Nix in the build system Bring the Android debug tools into the build via the Bazel-pinned nixpkgs (25.05), no ad-hoc SDK install: - @android_tools (//pi/hitl/phone:adb) — adb / platform-tools. Cross-platform; VERIFIED building AND running on the aarch64 container (Android Debug Bridge 1.0.41). - @android_emulator (//pi/hitl/phone:android_emulator) — an androidenv composed SDK (emulator + Google-APIs system image + cmdline-tools) from pi/hitl/phone/nix/ android-emulator.nix, native-ABI system image per host (arm64-v8a on Apple Silicon, x86_64 on x86_64). The Android SDK ships the emulator for macOS (Intel + Apple Silicon) and linux-x86_64 but NOT linux-aarch64 (the derivation reports "no sources for os=linux, arch=aarch64"), so the target is target_compatible_with {macOS, linux-x86_64} and cleanly skipped as incompatible on linux-aarch64 (this container). Also tagged manual (a multi-GB SDK download only this lane needs) so it stays out of `bazel build //...`. launcher.py resolves adb/emulator/avdmanager from the Nix SDK ($ANDROID_SDK_ROOT, PATH fallback), creates the AVD from the bundled system image, boots the emulator headless, and opens the served app in the emulator at 10.0.2.2 (its host-loopback alias — the mock/rig device URL is rewritten to it too). phone_e2e gains the --android branch. Verified (aarch64): `bazel build //pi/hitl/phone/...` green (emulator skipped); adb builds + runs; explicit emulator build is cleanly incompatible (not a nix failure). Booting the emulator needs a station with a hypervisor — an Apple-silicon Mac (which can host BOTH the Android and iOS lanes) or a linux-x86_64 box with /dev/kvm. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: custom Nix derivation for the arm64-Linux Android emulator Upstream nixpkgs' androidenv ships the emulator for macOS (Intel + Apple Silicon) and linux-x86_64 but NOT linux-aarch64 ("no sources for os=linux, arch=aarch64") — yet the arm64-Linux emulator exists (Asahi etc.). So provide it via a custom derivation, and widen the //pi/hitl/phone:android_emulator target to macOS + Linux (x86_64 + aarch64). android-emulator.nix: on aarch64-linux, compose the emulator-less SDK (cmdline-tools + arm64-v8a system image, arch-agnostic) and merge in a custom emulator built from Google's CI (ci.android.com — the only source). Reuses nixpkgs emulator.nix's exact buildInputs + autoPatchelf + LD_LIBRARY_PATH wrap. Sourcing it reproducibly is the subtle part: ci.android.com serves the emulator only via a TEMPORARY signed storage.googleapis.com URL (Expires=…&Signature=…) and garbage-collects old builds. A plain fetchurl can't pin that, so the source is a FIXED-OUTPUT derivation (pinned by content hash) whose builder scrapes the CI page, resolves the fresh signed URL, and downloads — verified working end to end here (scrape → signed URL → download → nix hash-check). The one thing that needs a maintainer/station: a CURRENT emulatorBuild + its hash (the checked-in example build is GC'd → NoSuchKey); outputHash is lib.fakeHash with a PIN MAINTENANCE note. Verified (aarch64): `bazel build //pi/hitl/phone/...` green (emulator `manual` → excluded); nix file parses; the fetch machinery downloads + hash-verifies. Booting still needs a hypervisor host (KVM / Hypervisor.framework). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: fix the aarch64 emulator fetcher — use the CI download API, not HTML scrape The previous fetcher grepped the ci.android.com artifact page for a storage.googleapis.com URL. That no longer works (and my earlier "verified end to end" claim was wrong): the current Artifact Viewer page ships `artifactUrl:""` and resolves the signed URL at runtime via the build API — the HTML contains no URL to grep. Replace it with the actual mechanism: androidbuildinternal.googleapis.com/.../builds/<id>/emulator-linux_aarch64/attempts/latest/ artifacts/sdk-repo-linux_aarch64-emulator-<id>.zip/url?redirect=true which 302-redirects to the temporary signed URL. Anonymous access is allowed for public builds — VERIFIED reachable here (the endpoint returns semantic 404s, not auth errors). `curl -fL` follows the redirect; a `PK` magic-byte check fails the build loudly on a purged/missing build instead of hashing a JSON error page. Also corrected, honestly: - The build id ships UNSET (`0`) rather than a purged example that looks real; the derivation fails loudly until a current `aosp-emu-master-dev` id + `outputHash` are pinned. - PIN MAINTENANCE now says the id must be read off the emulator grid IN A BROWSER: the build-LIST REST API is anonymously rate-limited + deprecated ("migrate to Build API v4"), and the grid renders its list via JS — so scripted discovery is unreliable (the *download* path is fine anonymously; only listing is walled). - Noted that dl.google.com's SDK channel carries only emulator-linux_x64 + emulator-darwin_ aarch64 (verified against repository2-*.xml) — its build numbers have no linux_aarch64 target, so they can't be reused as the pin. Verified (aarch64): nix file parses; `bazel build //pi/hitl/phone/...` green (emulator is `manual` → excluded). The emulator itself still can't build here until a live id is pinned. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: deep mapping journey — drive the real capture screen through decode The mapping journey only drove the start/stop-mapping RPCs; the synthetic camera → real detector → real decoder path was never exercised through the actual app. Now it is. - web/src/driver/guard.ts: a capture-controller registry (stats() + finish()) the capture screen populates under ?driver=. - web/src/ui/screens/capture.ts: under the driver, register that controller on mount / clear on unmount; publish live decode stats each frame; settle the harness's finish() when the solve completes / fails (incl. the "nothing to solve" + unmount paths, so it can't hang). - web/src/driver/harness.ts: openCapture (navigate to /capture — it owns start-mapping + the synthetic scene), captureStats / awaitDecode (block until N LED ids are recovered), finishCapture (end + VIO solve). finishCapture is timeout-bounded: solve() has no timeout of its own and the VIO solver can stall on a scene with no inertial data. - pi/hitl/phone/journeys/mapping_capture.json: connect → openCapture → awaitDecode, asserting every LED id is decoded end to end through the app (not the RPC, not a unit test). - pi/hitl/phone/phone_e2e.py + BUILD: serve the solver deployment (//solver:solver_web — wasm + worker) at /solver/ like the Pi server does, so the on-device solve path can load; merged serve root assembled via symlinks. - pi/hitl/phone/launcher.py: pipe the page console + uncaught errors to stderr — the only window into the in-page journey when headless (this is how the decode/solver issues were found). Verified (in-container, headless Chromium vs the mock): smoke + connect + config + mapping + mapping_capture all green; the synthetic camera decodes 30/30 LED ids through the real capture screen. The final VIO solve is wired (finishCapture) but left out of the default journey — the visual-inertial solver needs real motion/IMU the headless synthetic scene doesn't yet synthesize (planar fixture, no DeviceMotion); that synthesis is the follow-up. Web typecheck + 76 unit tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: synthetic IMU + parallax — the real VIO solve now converges mapping_capture proved decode; the full visual-inertial solve couldn't run because the synthetic scene had no inertial data (and solve() has no timeout, so it hung). Fix both ends: - web/src/xr/syntheticCaptureSource.ts: emit a synthetic IMU stream consistent with the pose track — body-frame angular velocity (so3_log of the rotation finite-difference) and specific force Rᵀ·(a_world − g) from the second difference of position, at 60 Hz, stamped in the frames' performance.now clock so the solver aligns inertial + visual by timestamp. Widen the camera path to the benchmark's radius-1.8 sweep for real parallax. This is a port of the solver's own synth.rs (its solvable canned problem), with the web geom/pinhole conventions (quatToRotMat = camera-to-world). The source now also serves as the capture screen's IMU flusher under the driver. - web/src/ui/screens/capture.ts: use the synthetic source as the imuFlusher under ?driver= (no DeviceMotion in a headless browser); driver-only solve diagnostics. - web/src/driver/harness.ts: finishCapture asserts the solve recovered ≥ half the fixture. - pi/hitl/phone/mock_device.py: reply `mapping_stopped` to stop_mapping{solveOnHost:false} — the phone-solve path awaits that ack, not `result_ready`. THIS was what wedged finishCapture (before the solve was even reached); not the IMU. - pi/hitl/phone/journeys/mapping_solve.json: connect → openCapture → finishCapture, asserting a solved map. Verified (headless Chromium vs the mock): the VIO solve converges — 960 detections + 516 IMU samples, gravity correctly ~9.8 m/s² in body-Y, progress 0→80% with reprojection RMS 1.0→0.6px, all LEDs recovered. All 6 journeys green (smoke, connect, config, mapping, mapping_capture, mapping_solve); web typecheck + 76 unit tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: real-BLE lane — software Improv peripheral (Bumble) + app toggle The real-BLE lane replaces the app-seam virtual-BLE mock with a real OS BLE stack pairing with a software Improv peripheral — so we exercise real pairing/provisioning, not the mock. - pi/hitl/phone/ble_peripheral.py: a Bumble Improv GATT server (the same service a real ESP32-C6 advertises). Mirrors web/src/net/improv.ts (UUIDs + wire) and virtualBle.ts's behaviour: on a wifi-settings RPC it answers RPC_RESULT with a redirect URL and steps CURRENT_STATE → PROVISIONED. run()/main() advertise it on a real Bumble transport (android-netsim / hci-socket / RootCanal) for the station. - pi/hitl/phone/ble_peripheral_test.py: a Bumble CENTRAL drives the full Improv handshake over a LocalLink — two complete BLE stacks (LL/L2CAP/ATT/GATT), no radio, no emulator. Verified in-container incl. the MTU-negotiation gotcha (the default 23-byte MTU truncates the redirect notification — real clients negotiate up) and the Improv error path. - App real-BLE toggle: `?driver=…&ble=real` makes the improv/bleTransport swaps fall through to real Web Bluetooth instead of the virtual mock (new driverUsesVirtualBle() predicate + setDriverBleMode in main.ts). - requirements: add `bumble` (+ transitive deps: cryptography, pyusb, …). BUILD: py_library `ble_peripheral`, py_test `ble_peripheral_test`, py_binary `ble_peripheral_server`. Verified (in-container): ble_peripheral_test green under Bazel (cryptography does NOT SIGILL under the hermetic interpreter here); web typecheck + 76 unit tests green; the virtual-BLE journeys (smoke/connect/config) still pass after the guard-predicate refactor. Station-only remainder (needs a hypervisor): boot the emulator, attach the peripheral over Netsim, and drive the Web Bluetooth chooser under automation — the peripheral + protocol (the portable, hard part) are done and CI-tested; the emulator boot + chooser is the spike. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * pi/hitl/phone: record that the arm64-Linux emulator is discontinued upstream Chasing a pinnable aosp-emu-master-dev build turned up a dead end: the branch is FROZEN — last green build 13278466 (2025-03-28) — and even that build's linux_aarch64 zip is garbage-collected (getdownloadurl → "attempt not found"; a maintainer confirmed the grid download 404s). No successor branch publishes a linux_aarch64 target, and the current released emulator has none either. So there is nothing live to pin from Google's CI. Update the PIN MAINTENANCE note (nix), the Android-lane section + Status (README), and the BUILD comment to say so plainly and point to the working path: run the emulator lane on a macOS (Apple Silicon/Intel) or linux-x86_64 station, where nixpkgs ships the emulator upstream with NO pin. Only a native arm64-LINUX station hits this gap, and its sole remaining route is building the emulator from source. The custom derivation + fetch machinery stay in place (still verified reachable) in case Google republishes; `emulatorBuild` stays UNSET so it fails loudly. No code change — docs/comments only. nix parses; `bazel build //pi/hitl/phone/...` green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Agent <k4757026@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fughilli
force-pushed
the
diffuse_capture
branch
from
September 22, 2026 03:07
b53363b to
d1922d2
Compare
fughilli
force-pushed
the
diffuse_capture
branch
from
September 24, 2026 03:58
d1922d2 to
c6b2061
Compare
On a diffused fixture, lighting every LED every frame blends adjacent spots (color bleed) and the diffuser lowers the luma derivative, so the blob detector under-segments and the per-LED hue decode is corrupted. Two-pronged fix. Stage A — model + validate. A headless image-space simulator (web/src/sim) renders real camera frames through an energy-conserving diffuser convolution (reproduces both color bleed AND the lowered luma derivative) and drives the PRODUCTION detector core (reducedToBlobs, extracted from detect.ts) + CvPipeline. A 2x2 ablation (web/tests/diffuse_sim) proves striding AND a local-contrast top-hat detector are each necessary and jointly sufficient: decode yield 0 / 0 / 0 / 1.0 across (all-lit,strided)x(default,adaptive) vs a clean 1.0 floor. Stage B — wire it (firmware-first), behind ?diffuse=1: - Stride schedule authority (web/src/code/stride.ts, mirrored in the Rust pattern crate; pinned by a shared golden). S uniform coverage phases + S-1 sparse bridge phases => per-phase spacing >= S, full coverage, and a depth-2 registration star so partial per-phase maps fuse with bounded error. - Protocol: stride_spacing / anchor_density / stride_phase on StartMappingOptions / ConfigureOptions / CodeParams (proto + JSON schemas + regenerated Rust/Python/TS/buf bindings). - Firmware player masks non-lit LEDs (returns BLACK, not None) and advances the phase on Configure, keeping spacing/anchor. - detect.ts GL fragment-shader top-hat prefilter, gated off by default. - Capture screen: ?diffuse=1&stride=N&anchor=M enables the prefilter + a low threshold and rotates the stride phase across epochs. Verified: web unit tests, firmware pattern/player tests, proto conformance, and the esp32c6 image builds. On-device tuning of S/A/sigma/gain/threshold against a real diffuser is pending. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fughilli
force-pushed
the
diffuse_capture
branch
from
September 28, 2026 19:09
c6b2061 to
8bae42d
Compare
Re-investigated the "firmware doesn't report hardware config" note: the hw-config report path IS present and reached in a matching-revision c6/c3 build (not target-gated) and seeds two channels from NVS at boot, so a fresh device reports >=1 channel and never 0 (#102 is an ancestor of HEAD, never reverted). The note appeared only because getHardwareConfig() had no deadline: a firmware too old to answer get_hardware_config (or a wedged link / mis-flashed image) left the request pending forever and the pre-rendered note stuck with no error. Not a firmware bug. Robustness (client.ts + hardwareSetup.ts): - request() takes an opt-in per-call timeout; getHardwareConfig uses it and rejects with a new RequestTimeoutError instead of hanging. Long-running calls (map solve -> result_ready) pass no timeout and keep waiting forever. - Hardware Setup now tells apart not-connected / firmware-didn't-answer (version- aware "out of date" wording, enriched with the welcome's fw version + commit) / answered-but-zero-channels, instead of one conflated "doesn't report" note. Capture-config UI, persistent (was only reachable via ?diffuse=1 and friends): - Inline diffuse/strided toggle on the capture screen, persisted to Behavior Settings so re-mapping never needs the URL param again. - Per-device stride on Hardware Setup (tracks a fixture's LED pitch/diffuser), persisted on the device record. - Anchor density, local-contrast gain, threshold, downscale, flip-V in Behavior Settings (app-global). - capture.ts resolves the effective config from these (per-device stride from the device; the rest from Behavior Settings); URL params remain per-run overrides. Tests: request timeout + first-settle-wins, per-device stride clamp/persist, diffuse params/toggle persistence. web typecheck + unit suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Diffuse-fixture LED capture
Maps LED fixtures that sit behind a diffuser. Today's capture lights every LED every frame and carries identity in hue — on a diffuser adjacent spots bleed together (merged blobs, washed chroma) and the diffuser flattens each spot's luma derivative, so the blob detector under-segments and the decode fails.
Two-pronged fix.
Stage A — model the problem, then beat it (fully verified)
A headless image-space simulator (
web/src/sim/render.ts) renders real camera frames through an energy-conserving diffuser convolution (reproduces both symptoms — color bleed and the lowered luma derivative) and runs the production detector core (reducedToBlobs, extracted fromdetect.ts) →CvPipeline.web/tests/diffuse_sim.test.tsasserts a 2×2 ablation:Clean (no diffuser) floor = 1.00 → striding and the adaptive detector are each necessary and jointly sufficient. The detector fix is a local-contrast top-hat (large-σ background subtract + low threshold = local adaptive detection), hill-climbed against the sim.
Stage B — wire it, firmware-first (behind
?diffuse=1)web/src/code/stride.ts, mirrored in the Rustpatterncrate, pinned by a shared golden):Suniform coverage phases +S-1sparse bridge phases → per-phase spacing ≥ S, full coverage, and a depth-2 registration star so partial per-phase maps fuse with bounded error.stride_spacing/anchor_density/stride_phaseonStartMappingOptions/ConfigureOptions/CodeParams(proto + JSON schemas + regenerated Rust / Python / TS / buf bindings).player: masks non-lit LEDs (returns black, not None) and advances the phase onConfigure, keeping spacing/anchor.detect.tsGL fragment-shader top-hat, gated off by default.?diffuse=1&stride=N&anchor=Menables the prefilter + low threshold and rotates the stride phase across epochs.Try the preview
Open the capture screen with
?diffuse=1&stride=4&anchor=3(tune with?lcgain=,?threshold=). Without the flag, behavior is unchanged (stride params unset ⇒ legacy all-lit).Verified
//web:unit_tests(incl.stride+diffuse_sim),//firmware/pattern:golden_stride_test(Rust↔TS parity),//firmware/player:session_test(masking + phase rotation),//shared/protocol/rust:conformance_test, web app typecheck, and//firmware/player_app:esp32c6builds. On-device tuning of S/A/σ/gain/threshold against a real diffuser is the remaining step. Native-iOS diffuse and Pi-player diffuse are follow-ups.🤖 Generated with Claude Code