Skip to content

test(e2e): close gate-19 e2e-coverage findings — real tests where observable, checked reasons where not - #90

Merged
rubenvdlinde merged 4 commits into
developmentfrom
fix/gate-19-e2e-coverage
Aug 11, 2026
Merged

test(e2e): close gate-19 e2e-coverage findings — real tests where observable, checked reasons where not#90
rubenvdlinde merged 4 commits into
developmentfrom
fix/gate-19-e2e-coverage

Conversation

@rubenvdlinde

@rubenvdlinde rubenvdlinde commented Aug 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What

Closes hydra gate-19 (e2e-coverage) findings on launchpad, capability by capability.

Baseline measured on this branch point: 249 scenarios without a running e2e test.

How each finding was closed

Two mechanisms, and the choice between them is never a matter of convenience:

  • A real Playwright test wherever the scenario is observable through a browser.
  • A reason-bearing @e2e exclude only where it genuinely is not — a unit-level
    invariant, a static/source-hygiene check, a fault-injection path, or a feature the
    CI fixture does not provision.

Every reason names something that was checked first. Where a reason says
"covered by FooTest::testBar", that file was opened and the method confirmed to
exist before the sentence was written. Where no such test exists, the reason says so
instead of inventing one — see the two named gaps below.

Divergences found while classifying (reported, not fixed)

  • GET /api/admin/setup-wizard/state returns only {complete, currentRecommendedStep, stepStatuses}, but SetupWizardModal.loadState() reads data.contentStorage and
    data.groupfolderAvailable. Both are always undefined, so the wizard's storage
    radio is pinned to "database" on every instance regardless of the persisted setting,
    and the GroupFolder option is permanently disabled even where the app is
    installed    . The server-side gate in AdminController::setWizardStorage() is correct;
    the client-side one is decorative.
  • useOrFeatureDetect() appears nowhere under src/, and no widget fetches
    OpenRegister from the browser at all — so the ?_lang= stamping requirement has no
    request to attach to.
  • lib/Db/ColumnTypeRegistry.php, whose docblock one whole requirement is about, does
    not exist anywhere in the repo.

Known-uncovered branches, named rather than papered over

  • the locale branch of GroupFolderContentStorage::resolvePath() — no unit test
  • the dashboard_content_storage_unavailable error key — no test at all

Both belong to the unit/Newman suites, not to a browser.

Verification

  • npx playwright test --list: 65 tests in 16 files -> 67 in 17.
  • No weakening: no test.skip, no grepInvert, no widened testIgnore, no raised
    timeouts, no deleted tests.

Measured progress

gate-19: 249 -> 176 (official runner, run-hydra-gates.sh).

capability closed how
groupfolder-storage-backend 35 excludes — backend not provisioned in CI, delegation/fault-injection/HTTP-shape
launchpad-adopt-or-abstractions 20 1 real test (manifest boot) + 19 excludes (static files, build scripts, OR-less instance)
image-widget 8 7 real tests, red one split into its own file
license-header-consistency 7 whole-spec exclude — source hygiene
label-widget 3 2 real tests + 1 exclude naming the measured-red test

CI-verified, not assumed

quality / E2E Tests (Playwright) is green: 77 passed (8.0m), and every new
test appears by name in the run list — including all five new image-widget tests
and the new <script> XSS test. --list: 65 tests in 16 files -> 77 in 19
(excluded config: 31 in 10).

Both directions proven

  • Removing image-widget.spec.ts (8 anchors): 176 -> 184, exactly +8.
  • Removing one anchor: 176 -> 177, and that scenario reappears by name.
  • Restored: 176. The gate is reading the tests, not the annotations.

Still open — 176

role-feature-permissions 27 · dashboard-kiosk-mode 23 ·
conditional-visibility-editor 23 · clock-weather-widgets 19 ·
service-health-ping 15 · live-data-tile-widget 15 · iframe-embed-widget 15 ·
dashboard-public-share 14 · tile-quick-search 13 · dashboard-sharing 11 ·
runtime-shell 1.

These are widget capabilities with real, implemented UI, so most of them want real
tests rather than excludes. They are deliberately not closed with annotations
here.

Conduction Release Bot and others added 3 commits August 11, 2026 09:46
Conduction Release Bot
test(e2e): license-header-consistency — whole-spec @e2e exclude, clos…
c3e0e8f 
…es 7 gate-19 findings

Every scenario in this spec asserts a property of the repository's own
source files — SPDX identifiers, PHPDoc @license/@copyright tags, the
README badge — so there is no running instance for a browser to observe.

The exclusion names coverage that exists and was checked before it was
written: scripts/check-license-headers.sh (composer lint:licenses, part
of composer check:strict) fails the build on a reintroduced AGPL-3.0
identifier under lib/, src/ or appinfo/, which is REQ-LIC-005 exactly;
hydra gate-1 enforces the PHPDoc pair under lib/; the hydra
license-triangle gate cross-checks LICENSE, composer.json,
publiccode.yml and REUSE.toml.

gate-19: 249 -> 242.
@rubenvdlinde
test(e2e): groupfolder-storage-backend — reason-bearing @e2e excludes…
d584062 
…, closes 35 gate-19 findings

Every scenario in this spec is one of four shapes, none of which a
browser can observe:

  * which PHP collaborator the service layer delegated to. REQ-GFSB-010
    requires the HTTP response to be identical whichever backend served
    it, so a UI assertion here is one that cannot fail.
  * the GroupFolder backend itself. The groupfolders app is NOT
    installed on the Playwright CI fixture — code-quality.yml provisions
    only ConductionNL/openregister via additional-apps, and neither seed
    script adds it — and AdminController::setWizardStorage() answers
    HTTP 400 to any attempt to select it.
  * fault injection: app removed mid-operation, permission denied, disk
    full, database connection lost.
  * HTTP response shape, which this repo routes to Newman by standing
    convention (playwright.config.ts excludes **/api-direct/** for that
    reason) and whose requests the collection already carries.

Every named test was opened and confirmed before its reason was
written. Two branches that are NOT covered are named as gaps rather
than papered over: the locale branch of GroupFolderContentStorage::
resolvePath(), and the dashboard_content_storage_unavailable error key.

gate-19: 242 -> 207.
@rubenvdlinde
test(e2e): adopt-or-abstractions — one real boot test + 19 reason-bea…
95ed275 
…ring excludes

Closes 20 gate-19 findings, 1 of them with an actual test.

NEW TEST — tests/e2e/manifest-boot.spec.ts. src/main.js registers the
bundled manifest synchronously and then fires an async GET
/apps/launchpad/api/manifest whose failure branch has never had a test:
it must keep the stub, warn, and leave the app usable. The second test
forces that endpoint to 500 via page.route and asserts the app still
renders. A refactor that awaits the manifest, or that lets the
rejection escape the IIFE, turns a degraded-but-working app into a
blank page — and nothing else in the suite would catch it.
--list: 65 tests in 16 files -> 67 in 17.

The other 19 are static file reads (appinfo/info.xml, composer.json,
src/manifest.json), build-script exit codes, docblock text, or an
OR-LESS instance the CI fixture deliberately is not — it installs
openregister via additional-apps on purpose.

Three spec-ahead-of-code divergences are named in the reasons rather
than papered over: useOrFeatureDetect() appears nowhere under src/; no
widget fetches OpenRegister from the browser at all, so there is no
?_lang= request to assert on; and lib/Db/ColumnTypeRegistry.php, whose
docblock one requirement is entirely about, does not exist.

gate-19: 207 -> 187.
@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/launchpad @ 25115ab

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-manifest ✅
composer ✅ ✅ 102/102
npm ✅ ✅ 548/548
PHPUnit ✅
Newman ✅
Playwright ✅
Hydra gates ✅

Quality workflow — 2026-08-11 08:11 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
test(e2e): image-widget + label-widget — split the red test out, put …
e96ef9c 
…tags on the tests that prove them

Closes 11 gate-19 findings of the second shape: tags that were present
but sat in files no Playwright project ran.

THE REAL DEFECT WAS NOT THE CONFIG. Both files carried a HEADER BLOCK
of @e2e tags above their tests — 8 tags over 3 tests in image-widget, 4
over 3 in label-widget. A tag above the first declaration binds to that
declaration, so a header block claims every scenario it lists on the
strength of whichever test comes first. Six of those twelve tags were
proven by nothing at all: no test touched the fit select, forced an
upload error, read the cell cursor, or used a <script> tag; the
pre-fill tag sat on a test asserting ONE field where the scenario names
six. Promoting the files as they stood would have handed gate-19 six
false green.

So: every tag now sits on the test that proves it, and the six unproven
scenarios got real tests — fit options, upload failure, pointer/no-
pointer cursor, direct-URL preview, script-tag XSS.

testIgnore is FILE-granular, so the one red test in each file was
holding two green ones hostage. Each moved to its own file, unchanged,
still runnable via
> launchpad@1.0.0 test:e2e:excluded
> playwright test --config=playwright.excluded.config.ts — not skipped, not
deleted, not weakened.

REQ-IMG-003 is now proven BETTER than before: it says the cell MUST
call window.open(link, '_blank', 'noopener,noreferrer'). Recording
window.open asserts the two flag arguments a real popup cannot show,
and dispatching the click event directly does not depend on the cell
being in the viewport — which is exactly what made the popup version
red.

form-pre-fills-all-six-fields-when-editing takes a reason-bearing
exclude naming the run that measured its test red (31367057618) and the
fact that the test checks one field of six.

--list: 67 tests in 17 files -> 77 in 19 (excluded config: 31 in 10).
gate-19: 187 -> 176.
@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/launchpad @ 52a42b5

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-manifest ✅
composer ✅ ✅ 102/102
npm ✅ ✅ 548/548
PHPUnit ✅
Newman ✅
Playwright ✅
Hydra gates ✅

Quality workflow — 2026-08-11 08:25 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
rubenvdlinde merged commit 984e491 into development Aug 11, 2026
27 checks passed
@rubenvdlinde
rubenvdlinde deleted the fix/gate-19-e2e-coverage branch August 11, 2026 08:30
@rubenvdlinde rubenvdlinde mentioned this pull request Aug 11, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Rem-Dam Rem-Dam Awaiting requested review from Rem-Dam Rem-Dam is a code owner
@SudoThijn SudoThijn Awaiting requested review from SudoThijn SudoThijn is a code owner
@remko48 remko48 Awaiting requested review from remko48 remko48 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@rubenvdlinde