fix(quality): a missing skip-discipline gate must fail, not warn - #595
Merged
Conversation
added 2 commits
August 27, 2026 18:17
Opting into e2e-skip-blocking was a NO-OP when hydra-gates was not vendored. The step warned and exited 0, so the job went green. Measured today on launchpad, which set e2e-skip-blocking: true and is the ONLY app in the fleet to enable it: ##[warning]check_e2e_skips.py not found (hydra-gates not vendored here yet) - skip discipline NOT measured on this run. The PR merged green. launchpad looked like the first app in the fleet enforcing skip discipline while measuring nothing at all. The REPORT branch a few lines below already gets this right, and its own comment names the stakes exactly: # An absent report is an absent measurement, never a pass. Said # loudly, because this is the one failure mode that would quietly # turn the whole gate into a no-op. That is precisely what happened -- the author guarded one of the two ways in. An absent GATE is an absent measurement for the same reason, so it now behaves the same way. `continue-on-error: !inputs.e2e-skip-blocking` on this step keeps it advisory for apps that have NOT opted in, so measure-only mode is unchanged. For an app that HAS opted in, a missing gate is now a failure whose message says how to fix it.
The gate could never find check_e2e_skips.py in ANY app, however
correctly hydra-gates was vendored.
conduction/hydra-gates is published from ConductionNL/.github itself,
whose composer.json declares:
"extra": { "hydra-gates": { "helpers": "hydra-gates/scripts/lib" } }
That path is relative to the PACKAGE ROOT, so composer installs the file
at:
vendor/conduction/hydra-gates/hydra-gates/scripts/lib/check_e2e_skips.py
^^^^^^^^^^^
The workflow looked for it WITHOUT the inner segment. Verified the file
is real and 14,546 bytes at hydra-gates/scripts/lib/check_e2e_skips.py
in this repo.
This is why the gate is enabled in zero of the 21 fleet repos, and why
launchpad -- which DOES require conduction/hydra-gates ^1.8.2 (lock:
v1.9.0) and DOES configure the html reporter in CI -- still reported
"hydra-gates not vendored here yet".
Without this, the fail-loudly change in the first commit would turn every
opted-in app red with no way to fix it. The two belong together.
Old paths kept as fallbacks so a checkout-style layout still resolves.
rubenvdlinde
added a commit
to ConductionNL/launchpad
that referenced
this pull request
Aug 27, 2026
) launchpad is the only app in the fleet with e2e-skip-blocking: true, and until now that flag did nothing: the E2E job printed ::warning::check_e2e_skips.py not found (hydra-gates not vendored here yet) - skip discipline NOT measured on this run and went green. The app looked like the first in the fleet enforcing skip discipline while measuring nothing. Two causes, both now fixed upstream: 1. check_e2e_skips.py existed only on .github main and was in NONE of the last 12 release tags. Cut as v1.10.0. 2. The workflow looked for it WITHOUT the inner hydra-gates/ path segment that composer actually installs it under. Fixed in ConductionNL/.github#595, merged. #595 makes a missing gate a hard FAILURE for an opted-in app rather than a warning, so without this bump launchpad's next run would go red -- the correct behaviour, and this is the fix it is asking for. Verified locally: composer update resolves v1.10.0 and the file lands at exactly the path the workflow probes: vendor/conduction/hydra-gates/hydra-gates/scripts/lib/check_e2e_skips.py 14546 bytes launchpad's last E2E run was 137 passed, 0 skipped, so the gate is expected to pass on real data -- which is the whole point of turning it on here first. Co-authored-by: Conduction Release Bot <release-bot@conduction.nl>
This was referenced Aug 27, 2026
Closed
rubenvdlinde
added a commit
to ConductionNL/launchpad
that referenced
this pull request
Aug 28, 2026
…#366) #354 turned this on. #356 removed it 16 minutes later while rewriting this file's triggers -- not a revert, just two edits landing together and one line not surviving. Worth restoring now for a better reason than the first time. When #354 merged, the gate was a NO-OP: it could not find check_e2e_skips.py and warned instead of failing, so launchpad looked like it was enforcing skip discipline while measuring nothing. Three fixes later it genuinely runs: - hydra-gates v1.10.0 cut, containing the script (#359 vendored it) - ConductionNL/.github#595 fixed the lookup path, which had omitted the inner hydra-gates/ segment composer installs under, and made a missing gate a hard failure instead of a warning On development's current head the gate EXECUTED and reported: 0/137 tests skipped (0.0%) across 30 spec files V1 spec files executing ZERO tests : 0 V2 skips deferring to a deploy state CI decides: 0 V3 skips/fixmes with no reason recorded : 0 Every skip names a real absence and every spec file ran something. That answers the open question from #354: tests/e2e/docs-screenshots.spec.ts does NOT trip the zero-test rule. So this is measured, not hoped for. launchpad passes on real data, and the flag can only keep it that way -- which is exactly what the input's own description asks for: flip it per app once that app is worked down. Co-authored-by: Conduction Release Bot <release-bot@conduction.nl>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opting into e2e-skip-blocking was a NO-OP when hydra-gates was not
vendored. The step warned and exited 0, so the job went green.
Measured today on launchpad, which set e2e-skip-blocking: true and is
the ONLY app in the fleet to enable it:
##[warning]check_e2e_skips.py not found (hydra-gates not vendored
here yet) - skip discipline NOT measured on this run.
The PR merged green. launchpad looked like the first app in the fleet
enforcing skip discipline while measuring nothing at all.
The REPORT branch a few lines below already gets this right, and its own
comment names the stakes exactly:
An absent report is an absent measurement, never a pass. Said
loudly, because this is the one failure mode that would quietly
turn the whole gate into a no-op.
That is precisely what happened -- the author guarded one of the two
ways in. An absent GATE is an absent measurement for the same reason,
so it now behaves the same way.
continue-on-error: !inputs.e2e-skip-blockingon this step keeps itadvisory for apps that have NOT opted in, so measure-only mode is
unchanged. For an app that HAS opted in, a missing gate is now a failure
whose message says how to fix it.