fix(proof): refuse empty-artefact stubs on the topic-VM path; real artefact in the wire check (Architecte follow-up 3/4)

[echobt]

Summary

Architecte / Owner follow-up 3 of 4 (empty-digest stub out). Based on main (merged with #248 / #252 / #251).

What happened on staging. The happy path went green on the digest of an empty tree: the RLM VM could not reach the artefact host (the host's ufw / Docker FORWARD chain drops by default whatever the per-VM allowlist says; the server sat in a container netns / on loopback the VM never sees), and the guest image substituted an empty artefact whose digest matched. Nothing in the repo could tell. Prod must never rely on that stub, so this PR makes it unable to pass at every layer and documents it as staging-only history.

Architecte lock (added in 42c62d65, on top of the P1 fix): the guest forwards the exact archive bytes it fetched — never an empty or substitute tree that still hashes, never a re-tar (exact bytes preferred over inventing a tree-digest scheme; exact bytes are possible here, so that is the contract); a fetch that fails or does not verify is RlmToHost::Failed (host HvError::Guest → agent 502 backend → CP 503, no row); the empty-digest stub stays refused (CP 400 on the digest of nothing, host refusal of content-less tars). Written into AGENTS.md item 7, docs/PROOF.md § Isolation boundary and the runbook's locked-by-design table; the host half is pinned by a_guest_whose_fetch_failed_answers_failed_and_nothing_is_scored (stand-in guest answers its Evaluate job Failed: no output, no sister jail, sister listener released, topic VM stays alive).

KVM host (proof-fc-host + proof-vm-proto)

• Artefact identity contract (Greptile P1, 9e99e381): the served file's bytes, verbatim. artifact_digest = sha256 of the exact file served at artifact_uri (an uncompressed tar of the recipe tree). The RLM guest fetches it, runs proof_vm_proto::tar::verify_artifact on the bytes as received against the job's digest, inspects a copy, and forwards those same bytes as SisterRequest.artifact_tar — it never re-tars (tar metadata / member order change under re-encoding, so a rebuilt tar hashes differently). The KVM host runs the same function in sister::check_request before any jail: shape (uncompressed ustar / GNU / pax with ≥ 1 byte of file content; ~110 LOC walker: checksum-verified headers, GNU base-256 sizes, L/K/x/g meta entries skipped), then digest. gzip, non-tar bytes, an empty archive, a tree of empty files, or bytes that do not hash to the paid digest are refused by name (artifact_tar hashes to …: the guest must forward the bytes it fetched from artifact_uri verbatim (a re-tarred tree never matches) …), logged as sister request refused: …, and the run comes back without an attestation → CP 503 for firecracker_required, no row. A digest that matches an empty tree is a guest whose fetch failed, not a miner's work. Test: the same files re-tarred (other order / other mtime) keep identical content and are refused; verbatim bytes verify.
• Guest contract doc (SisterRequest, artifact_tar, HostToMiner::Run, tar module, CustomRunRequest.artifact_uri): a failed or non-verifying fetch is RlmToHost::Failed, never a substitute, never a re-tar.
• Egress preflight (Greptile P2 fixed): at every VM boot the agent reads nft -j list ruleset and logs an advisory when a foreign forward chain has policy drop (ufw's / Docker's ip filter FORWARD, never our proof_vm_* tables) and its table has no rule accepting iifname pfc* — the documented ufw / DOCKER-USER fixes leave the policy at drop and add exactly that accept, so the line clears on the next VM boot once the fix is in. Never fails the boot; the runbook says connectivity is proven by § 5, not by the advisory.

Control plane (proof-http) — POST /v1/submissions answers 400 (no row) to an artifact_digest that is the sha256 of nothing (zero bytes, or the 10240-zero-byte empty tar): artifact_digest is the sha256 of empty input (or of an empty tar archive): hash the recipe bytes you ship at artifact_uri. Miner docs table + AGENTS.md item 5 + docs/PROOF.md updated.

Harness (proof-vm-wire-check.sh submit-probe) — a live run (--expect 2xx) can no longer start on a random digest. New --artifact-file F (sha256 computed here; must be an uncompressed tar with content — python tarfile in r: mode), --artifact-digest HEX, --no-fetch-check. Before posting it refuses the digest of nothing, empty / gzip files, loopback / .invalid URIs (the VM never sees the host's loopback), fetches --artifact-uri from the CP host and re-hashes it (mismatch or fetch failure = FAIL, no POST), and after a 2xx checks the row echoes the real digest. Fail-closed probes keep the random digest; matrix gains the digest-of-nothing 400 row.

Runbook — § 5 rewritten around a real artefact (tar -cf recipe.tar …, sha256sum, serve it on an allowlisted address, --artifact-file); new § Egress with the ufw (ufw-before-forward -i pfc+ -j ACCEPT in before.rules) and Docker (DOCKER-USER) accepts and how to verify; the fetch-fallback stub documented as staging-only and forbidden in the production RLM image; sign-off gains the real-digest + no-forward-advisory checks; env examples note the forward-policy trap.

No digest is invented anywhere (the two "digest of nothing" values are computed at runtime from zero bytes). No change to what a correct guest sends.

How to verify

cargo test -p proof-vm-proto -p proof-fc-host -p proof-http          # tar walker + verbatim-bytes identity (re-tar refused), failed fetch = failed job, refused sisters, CP 400
cargo test -p proof-vm-fc --test wire_check_script                  # harness: live probe refusals + fetch check (bash/curl/python3/tar)

# CP: the digest of nothing is a 400, no row (any scorable host)
deploy/scripts/proof-vm-wire-check.sh submit-probe --topic <custom-topic> --expect 400 \
  --artifact-digest $(sha256sum </dev/null | cut -d' ' -f1) --reason 'sha256 of empty input'

# harness: a live run refuses to start without the real bytes
deploy/scripts/proof-vm-wire-check.sh submit-probe --topic <t> --expect 201 --allow-live-run       # exit 2: needs --artifact-file
tar -cf recipe.tar -C <dir> recipe && deploy/scripts/proof-vm-wire-check.sh submit-probe --topic <t> \
  --expect 201 --allow-live-run --artifact-file recipe.tar --artifact-uri https://<host>/recipe.tar
#   PASS  the URL serves the declared bytes (sha256 … fetched + re-hashed here) → POST → … row carries the real artefact digest

# KVM host: journalctl -u proof-vm-orchestrator | grep -E 'accept no pfc\* tap|sister request refused'

Verified here: harness refusals (no digest / digest of nothing / empty tar / gzip / loopback URI / .invalid) exit 2 before any request; a local http server serving the wrong bytes → FAIL before the POST, the right bytes → PASS then POST; the proof-http test proves 400 + no row and that a real digest still scores on the same host.

Greptile

• Greptile has reviewed this PR; findings are fixed or answered (P1 preserve-archive-bytes + P2 forward-warning-never-clears → 9e99e381; Architecte lock → 42c62d65; threads answered and resolved)
• If the bot was silent, I commented @greptileai review

Test plan

• cargo test --workspace (isolated target dir)
• cargo fmt --all -- --check
• cargo clippy --workspace --all-targets -- -D warnings, cargo deny check, xtask loc-cap (proof-fc-host 1467 / proof-vm-proto 593) / consensus-lint / spec-check / design-check / external-docs-check

Risk

• Miner-facing: one new 400 (digest of nothing) — documented in docs/external-miner/proof.md; no real recipe is affected.
• Guest contract tightening: a guest that ships a gzip, content-less, or re-tarred artifact_tar is now refused (503, no row) instead of being run. The host always hashed the raw bytes against the paid digest, so a re-tarring guest was already broken silently; the contract now says so (forward the fetched file verbatim) and the refusal names the reason in the agent journal and the SisterAnswer::Refused error. Guest images built against proof-vm-proto should call tar::verify_artifact on the fetched bytes.
• No wire-format change (fields unchanged), no signature domain, no emission impact.

Naming

I did not rename BASE_* environment variables, deployed host paths
(/opt/base, /run/base, …), GHCR baseintelligence/base package names, or
base-*-v1 cryptographic domain tags, unless this PR’s purpose is a coordinated
cutover documented in docs/NAMING.md.

  

[echobt]

@greptileai please review empty-artefact stub refusal (Architecte follow-up 3/4) after rebase onto main post-#252.

[echobt]

@greptileai review again — previous Greptile check still pending after CI green.

[greptile-apps]

Greptile Summary

The follow-up retains the archive identity contract: the guest and host verify the exact uncompressed archive bytes served at the artifact URI against the submitted digest, rejecting substitutions and re-encoded archives. The networking advisory now recognizes TAP ingress accept rules in the affected nftables table, allowing the documented remediation to clear the advisory.

Confidence Score: 5/5

Safe to merge; no outstanding blocking issues remain.

There are no accepted new findings and no outstanding previous findings. The archive-bytes finding is fixed: the host verifies verbatim archive bytes against the paid digest before it creates a sister jail. The forwarding-advisory finding is fixed: the check recognizes a TAP ingress accept rule in the table that contains the dropping forward chain.

_{Reviews (3): Last reviewed commit: "test(proof-fc-host): a failed fetch is a..." | Re-trigger Greptile}

[echobt]

@greptileai please re-review after archive-bytes P1 fix (42c62d6).