Skip to content

harden scaffold-sync: tier-2 CI coverage gate for core exports #865

Description

@vivek7405

Problem

Scaffold-sync enforcement had only tier 1: the .claude/hooks/require-scaffold-with-src.sh commit floor, which blocks a packages/(core|server|cli)/src commit that stages NO scaffold surface. It proves a feature commit touched a scaffold file, but cannot tell a real runnable demo from a doc bullet. That is exactly how #848 shipped forbidden() / unauthorized() documented (app-tree bullets in the scaffold rule files) but with no gallery demo: the floor was green.

Tests are enforced two ways (a commit gate that says "include a test" AND CI that actually runs it). Scaffold-sync was missing that second, un-skippable tier.

Design / approach

Add tier 2 as a CI test that runs under npm test: reconcile the LIVE @webjsdev/core export surface against a hand-curated manifest, and FAIL when a new export is neither demoed in the gallery nor consciously exempted. A new export then turns CI red until it is classified, the analogue of "a test must exist AND pass." Classification is { demo } (a gallery file that references the symbol) or { exempt } with a reason (internal: plumbing, deferred: agent-facing but not yet demoed). The deferred backlog is tracked separately in #859.

Implementation notes (for the implementing agent)

Implemented in PR #861 (this issue retro-tracks it). Key landmarks:

  • Gate: test/scaffolds/gallery-coverage.test.js. A pure reconcile(liveExports, manifest, demoFileExists, demoFileTeaches) returns error strings; failure modes proven with synthetic inputs beside the real-surface assertion.
  • Manifest: test/scaffolds/gallery-coverage.json (100 exports: 12 demoed, 59 internal, 29 deferred).
  • Discovery: scripts/run-node-tests.js walks test/**, so the file runs in CI with no wiring.
  • Landmine: the demo check is a word-boundary match on the symbol, so a demo that only mentions an API in prose still fails unless it truly imports/uses it. render (client entry) and same-named server helpers (cache, revalidate) are exempt, not demoed.
  • Docs surfaces updated: the webjs-scaffold-sync skill (two-tier enforcement section + change-type row), the require-scaffold-with-src.sh header, and agent-docs/framework-dev.md.
  • Respect the prose-punctuation invariant (release: bump core/server/cli versions, honest engines fields #11) in all new text.

Acceptance criteria

  • A new/renamed @webjsdev/core export fails CI until demoed or exempted
  • Counterfactuals prove the gate fires (new export, stale key, missing demo file, prose-only demo, empty reason)
  • Manifest classifies exactly the live export surface (no missing, no extra)
  • notFound converted from prose to a runnable demo (/features/routing/missing renders 404)
  • Enforcement docs updated (skill, hook header, framework-dev.md)
  • Deferred backlog tracked in scaffold: close the coverage-gate deferred gaps (core + server + convention demos) #859

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

  • Status
    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions