research: local CI as the default gate (Rails 8.1 bin/ci) with cloud CI optional

[vivek7405]

Question

Rails 8.1 made "Local CI" a first-class feature: a generated bin/ci runs a declared step list on the developer's machine, and cloud CI became the optional half. Should WebJs adopt the same posture, and what would the WebJs-shaped version look like?

Conclusion

Yes. Ship a webjs ci command that runs a step list declared in the package.json webjs.ci block (the #550 orchestration seam, next to webjs.dev and webjs.start), scaffold a default block into every new app, and make the scaffold's existing GitHub workflow consume that same command so the step list has one source. Cloud CI stays generated by default (as Rails does) with a --skip-ci opt-out on webjs create. Signoff (gh signoff) is an opt-in flag, off by default, exactly as Rails ships it. Implementation is separate tracked work (see the follow-up issue linked in the comments once filed).

What Rails does (verified in the local clone at frameworks/rails, HEAD 52fa23ce8e, 2026-09-10)

Landed as "Structured CI with bin/ci" (rails/rails#54693, 2025-03-08, Rails 8.1), parallel groups added in rails/rails#56774 (2026-02-19).

The runner is ActiveSupport::ContinuousIntegration (activesupport/lib/active_support/continuous_integration.rb, ~230 lines) plus continuous_integration/group.rb (~220 lines):

• CI.run(title, subtitle) { ... } sets ENV["CI"] = "true", evaluates the block, prints a green/red result line with total runtime, and aborts (non-zero exit) on any failure.
• step "Title", "cmd" (or step "Title", "bin", "arg", ... for system-style escaping) runs one command with inherited stdio, times it, prints ✅ Title passed in 2.11s / ❌ Title failed in 0.01s, and records the result.
• group "Name", parallel: N { ... } collects steps first, then runs them on N threads. Each parallel step's output is captured (PTY when available so colours survive, else Open3) to a temp log and REPLAYED atomically when the step finishes, so output never interleaves. A progress line (Checks (12s) - Style: Ruby | Tests...) refreshes at 10 Hz. A sub-group inside a parallel group occupies ONE slot and runs its steps sequentially (dependent steps stay ordered); a parallel sub-group is an ArgumentError.
• --fail-fast / -f aborts right after the first failed step (checked after every step and group).
• success? lets the config branch (used only for the signoff step), failure title, subtitle prints a red heading, heading / echo are the colour helpers.
• On failure with more than one result, a summary lists every failed step (↳ Tests: Rails failed).
• SIGINT is trapped per step / group to print which step was interrupted.

The generated app (railties/.../templates/bin/ci.tt + config/ci.rb.tt): bin/ci requires config/boot, aliases CI = ActiveSupport::ContinuousIntegration, and requires config/ci.rb. The default config/ci.rb:

CI.run do
  step "Setup", "bin/setup --skip-server"
  group "Checks", parallel: 2 do
    step "Style: Ruby", "bin/rubocop"
    step "Security: Gem audit", "bin/bundler-audit"
    step "Security: Importmap vulnerability audit", "bin/importmap audit"   # importmap apps
    step "Security: Brakeman code analysis", "bin/brakeman --quiet --no-pager --exit-on-warn --exit-on-error"
    group "Tests" do
      step "Tests: Rails", "bin/rails test"
      step "Tests: Seeds", "env RAILS_ENV=test bin/rails db:seed:replant"
      # step "Tests: System", "bin/rails test:system"
    end
  end
  # if success?
  #   step "Signoff: All systems go. Ready for merge and deploy.", "gh signoff"
  # else
  #   failure "Signoff: CI failed. Do not merge or deploy.", "Fix the issues and try again."
  # end
end

railties/test/application/bin_ci_test.rb pins that shape (the parallel: 2 group, each default step, the Tests sub-group, the commented signoff).

The cloud half. rails new STILL generates .github/workflows/ci.yml (plus dependabot.yml) by default; --skip-ci omits them (app_base.rb:111). That workflow does NOT call bin/ci: it fans the same checks out as separate parallel jobs (scan_ruby, scan_js, lint, test, system-test), each with its own runner, DB service container, and timeout-minutes: 15. The Getting Started guide's stated rule for any OTHER provider is "point your pipeline to bin/ci. This ensures consistent checks locally and in CI." So Rails' posture is: the developer's daily gate is local, GitHub gets its own fan-out by default, and both are removable.

Signoff is basecamp/gh-signoff, a gh extension. gh signoff posts a green signoff commit status on HEAD (refusing unless HEAD is pushed, so a signoff always names a commit GitHub has), gh signoff fail posts a red one, and gh signoff install adds a branch-protection rule requiring that status. Its README pitch is the whole rationale: dev machines are fast and already paid for, cloud CI is slow and rented, and a green commit status is all a merge gate needs.

What WebJs has today

• The scaffold is cloud-first with no local aggregate. webjs create emits .github/workflows/ci.yml (packages/cli/templates/.github/workflows/ci.yml, wired in create.js:661, Bun-flavoured by bunifyCi in runtime-rewrite.js) with four jobs: conventions (npm run check + npm run doctor), unit (test:server), browser (test:browser after a Playwright install), e2e (WEBJS_E2E=1 test:server after puppeteer-core + Chromium). Locally a developer runs check, doctor, typecheck, test:server, test:browser, and the e2e variant as SIX separate commands; nothing aggregates them, times them, or exits with one verdict.
• The gate was deliberately moved OUT of the local hook. Lighten pre-commit to convention-check only; move test gate to CI #174 / Move the test gate from pre-commit to CI (framework + scaffolded apps) #188 lightened .hooks/pre-commit to a main-branch block only and put the test gate in CI, on the grounds that a per-commit test run slowed commit-per-logical-unit and that a cloud gate cannot be skipped by --no-verify. Local CI does not reverse that: webjs ci is a command a developer (or agent) runs before pushing, not a commit hook. The pre-commit stays as it is.
• The orchestration seam already exists. Unify webjs dev/start/db with npm-script behavior via a declarative tasks config #550 put webjs.dev.before / webjs.dev.parallel / webjs.start.before in the package.json webjs block, read by packages/cli/lib/app-tasks.js (pure reader) and run by packages/cli/lib/run-tasks.js (runBeforeSteps: sequential, shell: true, inherited stdio, PATH prefixed with every ancestor node_modules/.bin, first failure aborts). Types live in packages/core/src/webjs-config.d.ts (WebjsDevTasks, WebjsStartTasks), the JSON Schema in packages/server/webjs-config.schema.json, the validator in packages/server/src/webjs-config-validate.js (which enumerates the known top-level keys, so a new ci key MUST be registered there or it is rejected as unknown). Tests in packages/cli/test/app-tasks and run-tasks show the injectable-spawn pattern.
• Every step is already a webjs subcommand (check, doctor, typecheck, test --server, test --browser, db migrate, vendor audit), and webjs check / webjs test both already support an agent-facing --json.

Design decisions

1. Config lives in package.json webjs.ci, not in a ci.ts file

Rails uses a Ruby file because a Ruby DSL is the natural shape there. WebJs' precedent (#550) is that run orchestration is DATA in the webjs block, validated by the schema, typed by WebjsConfig, and readable by every tool (doctor, the MCP server, the cloud-workflow generator) WITHOUT importing app code. A ci.ts would need to be loaded through the TS stripper on both runtimes just to learn the step list, and the only thing Rails' DSL expresses that JSON cannot is the if success? branch around signoff, which becomes a flag (--signoff) instead. Settled by the #550 precedent.

Shape (a plain string is shorthand for { "title": cmd, "run": cmd }, mirroring dev.before):

"webjs": {
  "ci": {
    "steps": [
      { "title": "Setup", "run": "webjs db migrate" },
      { "title": "Checks", "parallel": 2, "steps": [
        { "title": "Conventions", "run": "webjs check" },
        { "title": "Health", "run": "webjs doctor" },
        { "title": "Types", "run": "webjs typecheck" },
        { "title": "Security: dependency audit", "run": "npm audit --audit-level=high" },
        { "title": "Tests", "steps": [
          { "title": "Tests: server", "run": "webjs test --server" },
          { "title": "Tests: browser", "run": "webjs test --browser" },
          { "title": "Tests: e2e", "run": "webjs test --server", "env": { "WEBJS_E2E": "1" } }
        ]}
      ]}
    ]
  }
}

A step is string | { title, run, env? } | { title, steps, parallel? }. A group inside a parallel group runs its steps sequentially in one slot; a nested parallel is a validator error (the Rails rule). env exists so the e2e opt-in does not depend on a POSIX VAR=1 cmd prefix.

2. webjs ci [--fail-fast|-f] [--only <title>] [--json] [--signoff]

• Sets CI=true in every child's env (Rails parity; nothing in the framework keys on it yet, apps may). Refuses to run at a workspace root the way webjs check does (webjs check at the monorepo root reports 61 false violations #1301).
• Sequential steps inherit stdio. Parallel steps run up to N children at once with piped stdio captured to memory (or a temp file past a size cap), replayed whole when the step ends, under the same heading / ✅ / ❌ / timing format Rails prints. FORCE_COLOR=1 is set on captured children so node --test and web-test-runner keep their colours (Node has no PTY without a native dependency, and a native dependency is disqualified). The progress line renders only when stdout is a TTY, so a CI log never fills with \r.
• One green/red total line with the elapsed time; a failure summary naming every failed step; exit 1 on any failure; SIGINT names the interrupted step(s) and exits.
• --json emits { ok, seconds, steps: [{ title, ok, seconds, group }] } for the agent loop, the same reason check --json exists.
• --signoff runs gh signoff after a fully green run and prints Rails' "CI failed. Do not merge or deploy." heading otherwise. It fails loudly when gh or the extension is missing. Off by default.
• When GITHUB_STEP_SUMMARY is set, the per-step table is appended there so a cloud run shows the same summary in the Actions UI.

Files: packages/cli/lib/ci-config.js (pure reader + normaliser, like app-tasks.js), packages/cli/lib/ci-runner.js (pure runner with injectable spawn, like run-tasks.js; envWithLocalBin exported from run-tasks.js and reused), the case 'ci' + usage entry in packages/cli/bin/webjs.js, WebjsCiConfig in webjs-config.d.ts, the schema, and the validator's known-key list.

3. The scaffold emits both halves, local as the daily gate, cloud consuming the same list

• create.js writes the default webjs.ci block above and a "ci": "webjs ci" script. Bun apps keep the plain webjs ci (it is tooling, like test / check), and runtime-rewrite.js swaps the audit step for Bun's audit command (verify the exact command against the pinned Bun before scaffolding it).
• .github/workflows/ci.yml collapses to ONE job: checkout, setup-node (or setup-bun), install, Playwright Chromium install, the DATABASE_URL: file:./ci.db env, then npm run ci. The step list stops being duplicated between the workflow and the developer's machine, which is the Rails guide's rule for every provider. The cost is that per-layer required status checks go away (one CI check instead of four); a team that wants the fan-out back uses a matrix job over webjs ci --only "<group title>", and the workflow comment says so.
• webjs create --skip-ci omits .github/workflows/ci.yml (and the PR template), matching rails new --skip-ci. The default stays "generate it", because Move the test gate from pre-commit to CI (framework + scaffolded apps) #188's rationale (a gate a local --no-verify cannot skip) still holds for teams that merge on GitHub.
• The pre-commit hook is unchanged; its comment gets one line pointing at npm run ci as the pre-push gate.
• .agents/rules/workflow.md "Every code change" item 4 becomes "npm run ci must pass" with the individual commands demoted to an explanation of what it runs; the scaffold AGENTS.md and the framework's own agent surfaces (AGENTS.md CLI reference, references/testing.md, references/built-ins.md config-block list) follow.

4. In-repo apps dogfood it

gallery, examples/blog, and website each get a webjs.ci block, and the repo's apps CI job (.github/workflows/ci.yml line 561) can run npx webjs ci per app instead of its hand-rolled per-app step list. The framework root itself is a workspace, not an app, so webjs ci refuses there and the root npm test matrix stays as it is.

Explicitly out of scope

• Reinstating any test run in the pre-commit hook (Lighten pre-commit to convention-check only; move test gate to CI #174 decided that).
• A cloud-side per-step fan-out by default (the Rails workflow does this; WebJs' single-source rule wins, with --only as the escape hatch).
• Making signoff the default merge gate. It is an optional flag, documented with gh signoff install.
• Windows-shell portability beyond what dev.before already promises.

Verification plan for the implementation

Unit: packages/cli/test/ci-config (shorthand normalisation, nested-parallel rejection, env merge, unknown-key rejection through the validator) and packages/cli/test/ci-runner with a fake spawn (order, slot count never exceeded, sub-group sequential inside a parallel group, fail-fast stops after the first failure, exit code, output captured and replayed once and whole, CI=true and the local-bin PATH on every child, --only selection, --json shape). Spawn-based: test/cli/ci.test.mjs running the real binary in a temp app. Scaffold: test/scaffolds/scaffold-integration.test.js asserts the webjs.ci block, the ci script, the single-job workflow, and --skip-ci. Bun: test/bun/ci.mjs runs the same temp app under bun. Docs: the api-coverage and gallery-coverage guards pick up the new config key and command.

[vivek7405]

Deep dive: the calls that could have gone the other way

Why not a ci.ts DSL like Rails' config/ci.rb? WebJs already has app-root convention files (env.ts, instrumentation.ts), so a ci.ts default-exporting a builder would fit the file conventions. It lost to the webjs block on three counts. The #550 decision put dev/start orchestration in JSON precisely so webjs dev and npm run dev share one primitive with no app code loaded; a ci.ts would have to be stripped and imported (Node and Bun, both strippers) to learn the step list. Every other reader (the schema, webjs doctor, the MCP server, the workflow) gets the list for free from JSON. And the one thing the Ruby DSL uses code for, the if success? branch around signoff, is one flag.

Why one cloud job instead of Rails' fan-out? Rails' generated workflow duplicates the step list (rubocop, brakeman, bundler-audit, tests each get a job) and the two drift independently; the Rails guide itself tells non-GitHub providers to just call bin/ci. WebJs' scaffold today has the same duplication (four jobs each re-stating what check, test:server, test:browser, and the e2e env do). A single npm run ci job removes it. The trade is per-layer required status checks, which webjs ci --only restores for a team that wants them.

Why keep generating the workflow at all, if local is the default? #188's argument still holds: a gate that runs where a local --no-verify cannot reach it. Rails also keeps generating it. The opt-out is --skip-ci, same flag name.

Why no change to the pre-commit hook? #174 decided that a per-commit test run fights commit-per-logical-unit. webjs ci is a pre-PUSH habit (and, with --signoff, a merge gate), not a commit hook. Nothing here reopens #174.

PTY. Rails captures parallel output through a PTY so child colours survive. Node has no PTY without a native module, which a buildless framework will not take on. FORCE_COLOR=1 covers node --test, web-test-runner, tsc, and drizzle-kit, which is every scaffold step.

CI=true. Set for Rails parity and because test tooling (Playwright, web-test-runner, npm itself) changes behaviour under it. Nothing in packages/* reads process.env.CI today; the docs should say the variable is set so an app can key on it.

Cross-references

• Structured CI with bin/ci rails/rails#54693 (Structured CI with bin/ci), Add parallel step groups to CI runner rails/rails#56774 (parallel groups), basecamp/gh-signoff.
• Lighten pre-commit to convention-check only; move test gate to CI #174 / Move the test gate from pre-commit to CI (framework + scaffolded apps) #188 (test gate out of pre-commit, into CI), Unify webjs dev/start/db with npm-script behavior via a declarative tasks config #550 (webjs.dev / webjs.start orchestration), dogfood: hybrid Tailwind in dev so local CSS never goes stale (follow-up to #947) #967 (dev.regenerate), feat: let doctor gate CI without making every warning fatal #1257 / feat: declare per-check doctor severity in webjs.doctor.gate #1296 (doctor.gate, the model for "CI fails on a declared subset"), webjs check at the monorepo root reports 61 false violations #1301 (webjs check refuses at a workspace root), Add a Bun-first scaffold mode (--runtime bun) across all 3 templates #541 / Flip bun-mode scaffold Dockerfile to a pure oven/bun:1 base #595 (Bun runtime flavouring in the scaffold).

[vivek7405]

Implementation is tracked in #1471, which carries the approved implementation plan (files, landmines, invariants, test layers, acceptance criteria).