Skip to content

fix(website): typecheck ratchet refuses a compiler that is not TypeScript - #1335

Merged
gHashTag merged 1 commit into
mainfrom
fix/typecheck-ratchet-real-tsc
Oct 4, 2026
Merged

gHashTag merged 1 commit into
mainfrom
fix/typecheck-ratchet-real-tsc

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 4, 2026

Copy link
Copy Markdown
Owner

npm run typecheck:ratchet passed falsely on a checkout without node_modules, and this makes it refuse instead.

What happened. The ratchet ran npx tsc. When typescript is not installed, npx resolves tsc to the unrelated npm package of that name. That package prints a banner ("This is not the tsc command you are looking for") and no diagnostics. The ratchet's only vacuous-pass guard was "no output means tsc did not run". The banner is output, so it got through, and the ratchet reported 0 errors across 0 files with 26 files improved and advice to run typecheck:update. Following that advice would have locked in a zero baseline from a compiler that never ran. Seen on 2026-10-04 while checking #1309 in a worktree without dependencies.

The fix, two guards:

  • The compiler is now the tsc of the typescript this package declares. It is resolved the way Node resolves an import from apps/website and run with node. When it is not installed, the ratchet exits 2 and says to run npm ci. It never falls back to a stand-in.
  • Output with not one diagnostic in it is refused. A clean tsc -b --pretty false prints nothing, so such output came from something else.

The existing guards are unchanged: empty output is refused, and so is a TS error that does not parse as a per-file diagnostic. The baseline, the per-file comparison and --update are unchanged too.

--selftest gains five cases, run in CI before the ratchet:

  • the captured fake-tsc banner, which must be refused;
  • empty output, which must be refused;
  • an unattributed error TS5083, which must be refused;
  • real diagnostics in two files, which must still be counted as {src/a.tsx: 2, src/c.ts: 1} (the negative control);
  • a temporary directory with no typescript, where no compiler may be found.

CI runs npm ci before the ratchet in both places that use it (website-checks.yml and deploy-site.yml), so CI behaviour should not change. The check on this PR's own CI run is what shows that.

{
  "version": 1,
  "head_sha": "db3069aa25f3bc7e482a44f4ed7373e51e1fa687",
  "summary": "The website typecheck ratchet no longer counts output from a compiler that is not TypeScript: it runs the tsc of the typescript package this project declares, exits 2 when that is not installed, and refuses output without a single diagnostic, which is what the unrelated npm package named tsc prints.",
  "changes": [
    "typecheck-ratchet.mjs resolves typescript/package.json from apps/website with createRequire and runs its bin/tsc with node, instead of running npx tsc.",
    "When typescript is not installed the ratchet prints that npm ci is needed and exits 2 rather than running anything else.",
    "Output parsing moves into readTsc, which also refuses output that contains no diagnostic at all, alongside the existing empty-output and unparsed-error guards.",
    "The --selftest mode gains five cases: the captured fake-tsc banner, empty output, an unattributed TS error, real diagnostics in two files as a negative control, and a directory with no typescript."
  ],
  "tests": [
    {
      "command": "node scripts/typecheck-ratchet.mjs --selftest (apps/website)",
      "status": "passed",
      "result": "10 of 10 self-test cases pass: the five existing comparison cases and the five new ones, exit 0.",
      "evidence": "Local run on db3069aa2."
    },
    {
      "command": "npm run typecheck:ratchet (apps/website, no node_modules)",
      "status": "passed",
      "result": "Refused: typescript is not installed here; run npm ci first. Exit code 2, where the old script reported 0 errors and passed.",
      "evidence": "Local run on db3069aa2 in a worktree with no apps/website/node_modules."
    },
    {
      "command": "--selftest against two mutants of typecheck-ratchet.mjs",
      "status": "passed",
      "result": "Both mutants are caught: dropping the no-diagnostics guard fails the fake-banner case, and returning a stand-in compiler path fails the no-typescript case, each with exit 1.",
      "evidence": "Mutants were edited copies of the script in a temporary directory outside the repository; the committed script was not touched."
    },
    {
      "command": "npm run typecheck:ratchet (apps/website, typescript 5.9.3 from the committed lockfile)",
      "status": "passed",
      "result": "179 errors across 26 files against a baseline of 179 across 26; no file gained type errors, exit 0.",
      "evidence": "Local run on db3069aa2. node_modules was a local directory of per-entry links to an existing install made from a byte-identical package-lock.json, so tsc wrote its build info into this checkout only."
    },
    {
      "command": "npm run typecheck:ratchet with one type error injected into src/data/blog/posts.ts",
      "status": "passed",
      "result": "180 errors across 27 files, src/data/blog/posts.ts 0 -> 1 reported as gaining a type error, exit 1.",
      "evidence": "Local run on db3069aa2; the injected line was reverted with git checkout afterwards."
    },
    {
      "command": "Website checks workflow (npm ci, --selftest, typecheck:ratchet)",
      "status": "not_run",
      "result": "Not run yet at the time of writing; the run on this PR is the authoritative check with a clean npm ci install.",
      "evidence": "The laptop disk was at about 99 percent, so a fresh npm ci was left to CI."
    }
  ],
  "limitations": [
    "A codebase with genuinely zero type errors would still be refused, because a clean tsc -b prints nothing; that is the existing behaviour and does not bite while the baseline is 179.",
    "The fake-banner case is a captured copy of the tsc package's output on 2026-10-04; a different wording of that banner is still caught by the no-diagnostics guard, not by matching the text.",
    "The local real-compiler runs used an existing install linked in entry by entry, not a fresh npm ci."
  ],
  "tags": [
    "Website",
    "TypeScript",
    "CI",
    "Verification"
  ],
  "blog": {
    "title": "The typecheck ratchet passed on a compiler that was not there",
    "summary": "Without node_modules, npx tsc ran the unrelated npm package named tsc, whose banner the ratchet read as zero errors; the ratchet now runs only the declared TypeScript compiler and refuses output that holds no diagnostic.",
    "outline": [
      "The symptom: a typecheck ratchet reporting zero errors across zero files and suggesting a new baseline, on a branch that had changed no TypeScript.",
      "The cause: npx resolving tsc to an unrelated npm package when typescript was not installed, and a guard that only refused empty output.",
      "The fix: resolve the compiler from the package's own dependencies the way Node does, and refuse output that contains no diagnostic at all.",
      "The self-tests, including the captured banner, a real-output negative control, and two mutants of the script that the tests catch.",
      "What is not covered: a genuinely clean codebase is still refused, and the local runs used a linked install rather than a fresh npm ci."
    ]
  }
}

🤖 Generated with Claude Code

…ript

The ratchet ran `npx tsc`. In a checkout without node_modules, npx
resolves `tsc` to the unrelated npm package of that name, which prints a
banner and no diagnostics. The banner was output, so the no-output guard
passed it, and the ratchet reported "0 errors across 0 files" with 26
files improved and advice to run typecheck:update, which would have
locked in a zero baseline from a compiler that never ran.

Two guards now:
- The compiler is the tsc of the typescript this package declares,
  resolved the way Node resolves an import and run with node. When it is
  not installed, the ratchet exits 2 and says to run npm ci.
- Output that contains not one diagnostic is refused. A clean tsc -b
  prints nothing, so such output came from something else.

--selftest gains five cases: the captured fake-tsc banner, empty output,
an unattributed TS error, real diagnostics in two files (negative
control: still counted), and a directory with no typescript.

Checked locally:
- --selftest: 10 of 10 ok.
- Without node_modules: exits 2 with the npm ci message.
- Mutations: dropping the no-diagnostics guard fails the banner case;
  returning a stand-in compiler fails the no-typescript case.
- With the real typescript 5.9.3 from the same lockfile: 179 errors
  across 26 files, equal to the baseline, exit 0.
- One injected type error: 180 across 27, exit 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the status:in-progress 🔵 Agent working label Oct 4, 2026
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

X Brain Health Check

Score: 100.0/100
Status: 🟢 HEALTHY
Threshold: 80/100

X Brain is above merge threshold

@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

🚨 CRITICAL: Brain Health Below Threshold

The brain health CI gate has failed. This PR cannot be merged until:

  1. All brain region unit tests pass
  2. Integration tests pass
  3. Stress test score >= 270/300
  4. Brain health score >= 80/100

Please review the failed job logs and fix the issues.

@gHashTag
gHashTag merged commit 460096c into main Oct 4, 2026
32 of 45 checks passed
@github-actions github-actions Bot added status:completed Done and removed status:in-progress 🔵 Agent working labels Oct 4, 2026
github-actions Bot added a commit that referenced this pull request Oct 4, 2026
fix(website): typecheck ratchet refuses a compiler that is not TypeScript (#1335)

The ratchet ran `npx tsc`. In a checkout without node_modules, npx
resolves `tsc` to the unrelated npm package of that name, which prints a
banner and no diagnostics. The banner was output, so the no-output guard
passed it, and the ratchet reported "0 errors across 0 files" with 26
files improved and advice to run typecheck:update, which would have
locked in a zero baseline from a compiler that never ran.

Two guards now:
- The compiler is the tsc of the typescript this package declares,
  resolved the way Node resolves an import and run with node. When it is
  not installed, the ratchet exits 2 and says to run npm ci.
- Output that contains not one diagnostic is refused. A clean tsc -b
  prints nothing, so such output came from something else.

--selftest gains five cases: the captured fake-tsc banner, empty output,
an unattributed TS error, real diagnostics in two files (negative
control: still counted), and a directory with no typescript.

Checked locally:
- --selftest: 10 of 10 ok.
- Without node_modules: exits 2 with the npm ci message.
- Mutations: dropping the no-diagnostics guard fails the banner case;
  returning a stand-in compiler fails the no-typescript case.
- With the real typescript 5.9.3 from the same lockfile: 179 errors
  across 26 files, equal to the baseline, exit 0.
- One injected type error: 180 across 27, exit 1.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant