Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,11 @@ on:
required: false
type: string
default: "[]"
e2e-skip-blocking:
description: "Make the E2E skip-discipline check a hard failure for this app. It reads the RUN's Playwright report — the only place a runtime `test.skip(cond, reason)` is visible — and reports three things: a spec file that executed ZERO tests, a skip deferring to a deploy state CI itself decides ('not deployed', 'deploy drift', 'predates'), and a skip carrying no reason. A skip naming a genuinely absent optional app or external service is allowed. Default false because eight apps carry findings today (measured 2026-08-24: 298/3210 tests skipped fleet-wide, 27 zero-test spec files) and a gate that turns them all red at once is a gate nobody can turn on — same reasoning as check-code-blocking. Flip it per app once that app is worked down; it can then only stay clean."
required: false
type: boolean
default: false
enable-coverage-guard:
description: "Enforce the coverage ratchet — READ-ONLY, never commits. On a PULL REQUEST the merge base is measured in the phpunit job and the change must not reduce coverage against it; the floor is measured, not typed, so it cannot go stale and there is nothing for an author to keep up to date. On a PUSH the committed .coverage-baseline is enforced as a conservative fail-safe floor (below it fails; above it is fine). PRs may raise that floor but never lower it. Requires scripts/coverage-guard.php supporting --against; the workflow probes for it and fails loudly rather than silently downgrading the check."
required: false
Expand Down Expand Up @@ -4856,6 +4861,70 @@ jobs:
SPEC_COVERAGE_JS
node /tmp/spec-coverage.js

# ── e2e skip discipline: read the REPORT, not the source ───────────────
#
# Every other e2e check in this file reads `tests/e2e/**` source. gate-19
# parses `test.skip(...)` statically and is careful about it. What no
# source reader can settle is the form the fleet actually uses:
#
# const present = await someLiveQuery(page)
# test.skip(!present, 'Members tab not deployed on this instance')
#
# At source level that is indistinguishable from a test that runs. Only
# the report of an actual run knows. Nothing read the report, so measured
# 2026-08-24 across 20 apps' latest GREEN `development` runs: 298 of 3210
# tests skipped (9.3%), 27 spec files executing ZERO tests, 61 skips
# deferring to a deploy state CI itself decides, and 142 carrying no
# reason at all. decidiq 29.8%, dossiq 27.7%, buildiq 26.3%.
#
# A zero-test spec file is still accepted by gate-19 as the `@e2e` anchor
# for an openspec scenario, so a scenario can be "covered" by a file that
# never runs.
#
# `!cancelled()` and not `always()`: a cancelled run has no verdict to
# explain, and a capture hung off a cancelled job produces nothing exactly
# when you need it. It deliberately runs when the suite PASSED — a green
# suite is where skips hide, and running only on failure would reproduce
# the blindness this step exists to remove.
#
# REPORT, NOT FAIL, on day one, exactly as app:check-code above: eight
# apps carry findings today, and turning them all red at once makes it a
# gate nobody can turn on. `e2e-skip-blocking` flips it per app as that
# app is worked down.
- name: E2E skip discipline
if: ${{ !cancelled() }}
continue-on-error: ${{ !inputs.e2e-skip-blocking }}
run: |
set -euo pipefail
GATE=""
for candidate in \
"server/apps/${{ inputs.app-name }}/vendor/conduction/hydra-gates/scripts/lib/check_e2e_skips.py" \
"hydra-gates/scripts/lib/check_e2e_skips.py"; do
if [ -f "${candidate}" ]; then GATE="${candidate}"; break; fi
done
if [ -z "${GATE}" ]; then
echo "::warning::check_e2e_skips.py not found (hydra-gates not vendored here yet) — skip discipline NOT measured on this run."
exit 0
fi

REPORT=""
for candidate in \
"server/apps/${{ inputs.app-name }}/playwright-report" \
"server/apps/${{ inputs.app-name }}/tests/e2e/playwright-report"; do
if [ -f "${candidate}/index.html" ]; then REPORT="${candidate}"; break; fi
done
if [ -z "${REPORT}" ]; then
# An absent report is an absent measurement, never a pass. Said
# loudly, because this is the one failure mode that would quietly
# turn the whole gate into a no-op.
echo "::error::no Playwright HTML report found — skip discipline could not be measured. Ensure the 'html' reporter is configured."
exit 1
fi

MODE=report
if [ "${{ inputs.e2e-skip-blocking }}" = "true" ]; then MODE=enforce; fi
python3 "${GATE}" --report "${REPORT}" --mode "${MODE}" --summary "${GITHUB_STEP_SUMMARY}"

- name: Upload Playwright report
if: always()
# `tests/e2e/…` is included because that is where the scaffolded config
Expand Down
Loading
Loading