ci(secret-scanner): make gitleaks gate for real, not continue-on-error - #500
Merged
Conversation
The gitleaks job carried continue-on-error: true (comment: gitleaks-action tool-cache PATH injection unreliable on self-hosted runners), so the estate's primary secret scanner could not fail a single PR anywhere it is used. Replace gitleaks/gitleaks-action with a pinned, checksum-verified gitleaks 8.18.4 binary run directly (gitleaks detect --source . --no-git --exit-code 1). A direct install is deterministic on every runner, removing the exact unreliability the continue-on-error was papering over, so the scan now gates. Comments/summary were already disabled via env vars, so the action provided only the scan; the swap loses no functionality and drops the now-unneeded GITHUB_TOKEN env and fetch-depth: 0. Verified locally with the pinned binary: checksum matches; a private-key fixture exits 1, a clean tree exits 0; actionlint clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
hyperpolymath
marked this pull request as ready for review
July 17, 2026 22:02
hyperpolymath
enabled auto-merge (rebase)
July 17, 2026 22:02
hyperpolymath
disabled auto-merge
July 17, 2026 22:32
This was referenced Jul 20, 2026
Closed
hyperpolymath
added a commit
that referenced
this pull request
Jul 21, 2026
## The main fix: standards' own secret scanning has not run since 2026-07-06 `secret-scanner.yml` pinned the reusable at `@891b1ed` (2026-07-06), which is **21 commits behind #500** (`08e2d246`). At that pin the reusable's `gitleaks` job requests `pull-requests: write` + `actions: read`, but this caller grants only `contents: read`. A called workflow cannot exceed its caller's grant, so **every run terminated at `startup_failure`**: ``` startup_failure main 2026-07-20T17:36 startup_failure test/signed-push-smoke 2026-07-20T17:33 startup_failure main 2026-07-20T17:24 startup_failure feat/signed-push-app-action 2026-07-20T16:55 startup_failure main 2026-07-20T13:43 startup_failure dependabot/github_actions/actions-5fa456a2a3 2026-07-20T13:28 ``` This is worse than a fake-green gate: the scan never executed at all, on the repo that *defines* the estate's secret-scanning standard. **Fix:** use the local relative path `./.github/workflows/secret-scanner-reusable.yml`, mirroring the `governance.yml` exemplar. This repo owns the reusable, so it should always scan itself with the current version — and a local path **cannot re-stale**, so this class of breakage cannot recur here. The current reusable needs only `contents: read`, so the existing grant is already correct (the in-file comment was accurate; only the pin was stale). ### Estate-wide corollary (not fixed here) The same `@d135b05` / pre-#500 pins are still in place on callers across the estate. Verified for the record: ``` $ gh api repos/hyperpolymath/standards/compare/08e2d246...d135b05 {"status":"behind","ahead":0,"behind":64} ``` At `d135b05` the gitleaks step is literally: ```yaml - name: Gitleaks Secret Scan continue-on-error: true # <-- cannot fail uses: gitleaks/gitleaks-action@e0c47f4f... ``` versus current `main`: ```yaml - name: Install gitleaks (pinned + checksum-verified) # sha256-verified binary - name: Gitleaks secret scan (gating) run: ... gitleaks detect ... --exit-code 1 # gates for real ``` So bumping those caller pins is **not cosmetic staleness** — it converts the estate's primary secret scanner from cannot-fail to actually-gating, *and* drops the permission demand to `contents: read` (removing the startup_failure hazard). That caller sweep is tracked separately. ## Secondary fixes - **`pages.yml`** (added in #503): had no SPDX header — tripping the governance "missing SPDX header" gate — and four unpinned tag refs, which the estate Actions policy rejects at run creation. Added `MPL-2.0` (**this repo's licence — deliberately not AGPL**) and pinned all four to release SHAs, keeping the current majors: `checkout` v7.0.0, `upload-pages-artifact` v5.0.0, `deploy-pages` v5.0.0. (Note: paint-type's pages.yml pins v3/v4 — copying those here would be a downgrade.) - **`spark-theatre-gate.yml`**: corrected a mislabelled pin comment — SHA `9c091bb2` is v7.0.0, not v4. Cosmetic, no behavioural change. ## Verification - All three files parse (`yaml.safe_load`). - The `startup_failure` claim is from `gh run list`, not inference; this PR's own Secret Scanner run is the direct test — it should execute rather than startup_failure. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
hyperpolymath
added a commit
that referenced
this pull request
Jul 21, 2026
…this repo only) + scorecard-verify false-red (new) (#514) Completes the run to green on `main`, after #511 (secret-scanner caller + SPDX) and #512 (gitleaks allowlist). Two remaining causes, independent of each other and of those PRs. ## 1. Hypatia has never run — the caller under-grants (#451) `hypatia-scan-reusable.yml` declares `security-events: write` (it uploads SARIF). `hypatia-scan.yml` granted `security-events: **read**`. A called workflow may never request more than its caller grants. When it does, GitHub rejects the run at **startup** — before any job exists: ``` completed startup_failure Hypatia Security Scan main push 1s $ gh run view 29764318317 --log-failed failed to get run log: log not found # <- there is no log; no job ran ``` Granting `read` did not degrade the scan to a no-upload mode. **It stopped the scan from existing.** The reusable carried this comment: ```yaml continue-on-error: true # callers granting only security-events:read skip gracefully ``` That is a false mental model, and it is load-bearing — it is the reason the mismatch looked intentional. Nothing skips gracefully in a job that was never created. Corrected in place. This is exactly the #451 pattern that startup-fails reusable-workflow callers estate-wide. **Fixed here on the canon's own caller only** — deliberately demonstrated on one repo before any propagation is proposed. ## 2. Seven "fake passes" that were every one of them real (#381) `registry-verify.yml` runs `build-scorecards.sh --verify`, which **executes** every pass-row's check. The job installed no tools, and `ubuntu-latest` ships neither ripgrep nor xmllint: | rows | mechanism | observed | |---|---|---| | `k9-svc/M6` | calls `xmllint` directly | exit **127** | | 6 × `release-pre-flight/*` | invoke `v1-audit.sh`, which hard-exits 2 without ripgrep, so the greps match nothing | exit **1** | The verifier reported all seven as **"claimed PASS but the pass is not real"**. Reproduced locally by stripping `rg` from `PATH` — identical output. Measured with the tools present: ``` ── verify: 87 grounded pass · 0 broken pass · 0 unrunnable · 0 self-asserted · 0 stale-fail EXIT=0 ``` **All seven claims were true.** The checks had never run. A false-red is not harmless: it trains maintainers to ignore a gate — the same end state as the fake-green gates #500 removed, reached from the opposite direction. Two changes, since installing the tools alone leaves the trap armed for the next missing dependency: - `registry-verify.yml` installs `ripgrep` + `libxml2-utils`. - `run_verify()` **refuses to judge pass-rows at all** in an incomplete environment, and reports genuinely unrunnable checks as `unrunnable` — still failing, since an unverifiable pass must never go green — rather than as false claims. ### The required-tool list is declared, not inferred An earlier draft parsed check strings to discover their commands. It misparsed quoted regexes (`"a\|b\|c"`) and `[ $(…) -ge 25 ]`, inventing **35** failures — precisely the sin the guard exists to prevent. Replaced with a measured, documented list: `xmllint` is named by 4 checks, `jq` by 1, and `rg` by none but is required transitively via `v1-audit.sh`. ## Verification - complete env → `87 grounded · 0 broken · 0 unrunnable`, exit 0 - incomplete env → fails fast, judges nothing, names the missing tool - `k9-svc/mime/k9.xml` + `k9.uti.plist` independently confirmed well-formed and the two companion files present — so installing xmllint turns that row **genuinely** green rather than exposing a real fault - all three workflow files parse ## Not included, on purpose `.gitleaks.toml` — **#512 covers it and is the better implementation.** I built one independently and then discarded it: my regexes matched against the whole line unanchored, and canary-testing showed that suppressed real tokens. | planted secret | gitleaks defaults | my draft | |---|---|---| | `ghp_…` PAT | caught | caught | | same PAT, line also contains `EXAMPLE_KEY` | caught | **suppressed** | | same PAT, line ends `# ...` | caught | **suppressed** | #512's anchored `^…$` design has no such hole. Verified independently against gitleaks 8.18.4 (the exact CI build, checksum-matched): all four adversarial fixtures still caught, and `0` findings on the tree. Adopt #512. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
hyperpolymath
added a commit
that referenced
this pull request
Jul 21, 2026
…e to real (#505) (#513) Promotes the two `governance-reusable.yml` jobs that **could not fail** into real gates — tasks 1, 2 and 3 of #505. ## Blast radius was measured first, not guessed Over the **412 real repo-root callers** of `governance-reusable` in the estate checkout (1,106 raw grep hits reduce to 412 once nested monorepo subdirs — whose `.github/workflows` GitHub never reads — are excluded): | check | missing | decision | |---|---|---| | README | **0 / 412** | **blocking now** — arming reds nobody | | LICENSE | **0 / 412** | **blocking now** — arming reds nobody | | CONTRIBUTING | 54 / 412 (13%) | warn → **blocking from 2026-08-21** | | no Guix/Nix packaging | 70 / 412 (17%) | warn → **blocking from 2026-08-21** | **Verified against all 412 callers with the final scripts: 0 red today on both gates.** The grace window buys time without manufacturing red noise now — and it is a *real date that flips itself*, not a flag defaulting to off (which would be the theatre this replaces). **94 unique repos** must land a `CONTRIBUTING` and/or packaging file before the cutoff. Full list attached as a comment below. ## What changed **`scripts/check-docs-presence.sh`** (new) — replaces the `::warning::`-only step. Split gate: README+LICENSE fail immediately, CONTRIBUTING warns until the cutoff. Critically, it **never prints a pass claim while a required file is absent** — grace mode prints `NOT YET ENFORCED`, not a green tick. **`scripts/check-package-policy.sh`** (new) — replaces the job that ended in an unconditional `✅ Package policy check passed`. Two honesty fixes beyond adding `exit 1`: - Predicate tightened from *"any `*.scm` anywhere"* (a stray Guile source file satisfied it) to a genuine packaging artefact (`guix.scm`/`manifest.scm`/`channels.scm`/`.guix-channel`). Tightening moved only **13** repos, so honesty was nearly free. - Vendored trees (`node_modules/`, `deps/`, `.lake/`, `vendor/`, `.git/`) are pruned, so a dependency can't satisfy the policy on the caller's behalf. - The `git diff HEAD~1` lockfile branch is **dropped, not stubbed**: it never resolved under `ref: github.sha` without `fetch-depth`, so it was inert. hypatia `cicd_rules/nodejs_detected` is the real rule. Broadening it beyond `package-lock.json` is follow-up, noted below. **Redundant trufflehog removed** (#505 task 3). It was `continue-on-error: true` so it never gated, and it duplicated the real gitleaks gate from #500. `secret-scanner-reusable.yml`'s header *already* claimed "Trufflehog removed as redundant" — this reconciles the estate with itself. Removing a never-failing step cannot newly-red any caller. **Both gates refuse to be disarmed.** A malformed cutoff would make the date comparison pick the grace branch forever — silently restoring the fake gate. Both scripts validate their own cutoff format and hard-fail on garbage. ## A real bug, found only by running against real repos `find … | head -1` under `set -o pipefail` is a **SIGPIPE race**: `head` exits after line 1, `find` keeps writing, takes SIGPIPE, and `pipefail` propagates it — aborting under `set -e`. It only bites on trees large enough that `find` is still running, so **all 20 unit fixtures passed while two real estate repos red intermittently for no policy reason**. Fixed, with a regression test that was itself red-teamed: it *fails* against the buggy form and passes against the fix. (The first version of that test was a false-green — 3,000 non-matching files don't trigger it, because `find` emits one line; the fixture now creates 4,000 *matching* paths.) ## Verification `just governance-gates-test` → **21 cases, all green**. Drives both gates on **both sides of the cutoff** (`DOCS_TODAY` / `PKG_TODAY` seams), plus alternate extensions, inclusive-cutoff boundary, vendored-tree pruning, stray-`.scm` rejection, malformed-cutoff refusal, and the SIGPIPE regression. This matters because **#505 notes standards CI does not exercise the reusable for SHA-pinned callers** — watching this repo go green proves nothing about them. Note however that **396 callers pin `@main`**, so they pick this up *immediately on merge*; the grace window is what makes that safe. ##⚠️ Reviewer note — the plumbing proof and the expected red standards calls this reusable **on its own PRs** (`governance.yml` → `./.github/workflows/governance-reusable.yml`), and the script checkout is pinned `ref: main`. The new scripts are not on `main` until this merges, so the final commit's governance run **will red for that reason alone, not because the change is broken**. To prove the plumbing (sparse-checkout → `cp` to `$RUNNER_TEMP` → `rm -rf .standards-checkout` → run) actually works end-to-end, this branch temporarily points `ref:` at itself; that run is the evidence, and the `ref:` is reverted to `main` before merge. See the commit history. ## Rollback Revert this PR, or re-pin callers to the previous `governance-reusable` SHA. The two new scripts are additive — nothing else calls them. ## Follow-up (not in scope here) - Broaden hypatia `cicd_rules/nodejs_detected` beyond `package-lock.json` (yarn/Gemfile/Pipfile/poetry lockfiles), recovering the coverage the inert branch nominally claimed. - The 94-repo remediation list before 2026-08-21. Refs #505, #500 --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
hyperpolymath
added a commit
that referenced
this pull request
Jul 27, 2026
…from reusable (#528) Restores `standards` own secret scanning, which has not executed since 2026-07-26. ## The outage Every recent `Secret Scanner` run on `main` is `startup_failure` — zero steps, no log, **no check run at all**. The repo secret scanning was silently dead, while a live write-scoped credential sits in the tree. ## Root cause `0ced540` ("chore: estate-wide security compliance", 2026-07-26) added `actions: read` to the **workflow-level** `permissions:` block of `secret-scanner-reusable.yml`. No job needs it — the only action used is `actions/checkout` (×3); there are no artifacts, no cache, no `gh api` calls, no workflow-run queries. Since #500 replaced `gitleaks-action` with a pinned checksum-verified binary, `pull-requests: write` is unnecessary too. `rust-secrets` and `shell-secrets` had no job-level block, so they **inherited** that unused grant. A called workflow cannot exceed its caller **job-level** grant, so the correctly least-privileged caller — granting `contents: read` alone — was rejected at startup. A sweep named "security compliance" disabled the security scanner. ## Fix — all three layers 1. **Workflow-level block reduced to `contents: read`**, restoring parity with the known-good pinned versions (`c65436ee`, `d135b05`). 2. **All three jobs declare `permissions: contents: read` explicitly** instead of inheriting. Inheritance is what let an unused grant leak into two jobs unnoticed; explicit blocks mean a future widening cannot silently re-break callers. 3. **The caller-example comment block is corrected.** It still instructed wrappers to grant `pull-requests: write` + `actions: read` — guidance that was correct only before #500, and which propagates to every wrapper minted from it. It now documents the minimal correct shape plus the two rules that make this failure mode invisible. Also removes the caller own unused `actions: read`, and the incorrect claim that moving to a local path meant *"this class of breakage cannot recur"* — it recurred by a different route. ## Prevents a latent mass outage **326** estate wrappers call this reusable; **156** grant `contents: read` only. They are green today *solely because they pin older, clean SHAs* (181 on `c65436ee`, 83 on `d135b05`). Re-pinning them to current `main` — as the planned pin-refresh sweep would do — **would have broken all 156 at once.** This must land before any such sweep. ## Verification - `yq` parse OK; effective grants confirmed `contents: read` at workflow level and on each of the three jobs. - `actionlint` clean on the caller. On the reusable the only finding is a **pre-existing, intentional** SC2053 (glob matching for `.shell-secrets-ignore`) — count identical before and after (1 → 1). - Sampled five `contents`-only estate callers on their current pins (`anamnesis`, `boj-server`, `echidnabot`, `conflow`, …): **all green**, confirming they are insulated rather than already broken. ⚠ **Draft pending the real proof:** the check that matters is this PR own `Secret Scanner` run going green. A rejected run produces no check run, so absence of red is not evidence — verify with `gh run list --json conclusion`. This exact failure mode has already survived one "fix". Refs #500. Fixes the `startup_failure` introduced by `0ced540`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
hyperpolymath
added a commit
that referenced
this pull request
Aug 7, 2026
…ly (#596) The estate gitleaks baseline could not be **inherited**, only **duplicated**. This adds the sharing mechanism that several repos already assume exists. ## The problem A consumer repo wanting the shared allowlist had two options, both bad: * **Copy it** — copies drift; a fix to one is a fix to none of the others. * **Push its repo-specific entries into the shared file** — which blinds all 400+ repositories. There was no third option, because the mechanism did not exist. **metadatastician/stapeln#21 is what surfaced it.** It ships a well-reasoned config doing `[extend] path = ".gitleaks-estate.toml"`, on the stated premise that this workflow stages that file. It does not, and never did: ``` FTL failed to load extended config, err: open .gitleaks-estate.toml: no such file or directory ``` That is worse than a missing feature. The config loads nothing, **the scan runs over no files at all**, and the failure presents as a findings failure — a scanner that appears to be working and is scanning nothing. ## What this adds | | | |---|---| | `config/gitleaks/estate-baseline.toml` | The shared baseline, `git mv`d out of this repo's root config so history is preserved. Only universally-true entries belong here. | | `.gitleaks.toml` (this repo) | Now a four-line consumer of it. **Deliberate dogfooding** — the mechanism is exercised by `standards`' own scan, so a change that breaks consumers breaks us first. | | `secret-scanner-reusable.yml` | Stages the baseline as `.gitleaks-estate.toml`, **only when the repo's config actually references it**. | | `docs/GITLEAKS-ESTATE-BASELINE.adoc` | How to opt in, and the two traps below. | **This is a no-op for every existing caller.** A repo that does not reference the baseline gains nothing in its scanned tree. ## Two things verified rather than assumed **1. `[extend] path` resolves against the process working directory** — not the config file's directory. Tested directly: the same relative path resolves from the repo root and fails from a subdirectory. That is why the baseline is staged at the workspace root, and it confirms the premise #21 was written against was correct. **2. Moving the baseline made it scannable.** As `.gitleaks.toml` gitleaks excluded it automatically as its own config. At its new path it is an ordinary file, and its documented placeholder shapes immediately tripped `generic-api-key` — a finding caused purely by renaming the file. Caught by scanning before and after: ``` before the move: 9 findings after the move: 10 findings <- the baseline flagging itself after allowlisting both paths: 9 findings <- zero regressions ``` Both the canonical path and the staged `.gitleaks-estate.toml` are now in the baseline's own `paths`. If it moves again, those entries move with it. ## Design notes **Pinned by construction.** The baseline is fetched at `${{ github.job_workflow_sha }}` — the commit of this reusable workflow — so a caller pinned to an old SHA gets that SHA's baseline. The workflow and the baseline it stages can never disagree, and there is no moving `@main` fetch. **The checkout is removed before scanning.** Otherwise `standards`' own tree is scanned as if it were the caller's, which is how a shared config turns into a wall of someone else's findings. **Failure is loud.** If the baseline is referenced but cannot be fetched, the job **errors** rather than scanning with a silently reduced config. A scan that quietly loses its allowlist produces a wall of false positives, and the usual response to that is to switch the gate off — which is exactly the history this file already records from before #500. ## Verification ``` gitleaks detect --source . --no-git # 9 before, 9 after — no regression python3 -c "import yaml; yaml.safe_load(open(...))" # workflow parses ``` Unrelated pre-existing local edits in this checkout were deliberately left unstaged; this PR touches four files. ## Follow-up Once this lands, stapeln#21 becomes viable as written — its estate-extension approach and its documentation discipline ("each entry names what the value is"; plant a canary and confirm it is still detected) are better than the `useDefault`-plus-local-paths approach currently proposed in stapeln#20. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ---- ## Summary by Gitar - **Documentation & Standards:** - Added `ZIGZAG-TESTING.md` and `ZIGZAG-TESTING.adoc` defining aspect-oriented testing methodology - Updated `TEST-NEEDS.md` to require Zigzag Testing for CRG Grade B/A <sub>This will update automatically on new commits.</sub> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Problem
The
gitleaksjob insecret-scanner-reusable.ymlcarriedcontinue-on-error: true:So the estate's primary secret scanner cannot fail a single PR anywhere this
reusable is consumed. The flag was a workaround for a real infra problem — the
gitleaks-actioninjects gitleaks into the runner tool-cache and prepends it toPATH, which is unreliable on self-hosted runners — but the workaround throwsout the signal with the flakiness: a genuinely leaked secret now passes green.
Fix
Replace the flaky action with a pinned, checksum-verified gitleaks 8.18.4
binary run directly, and let it gate:
A pinned direct install is deterministic on every runner — it removes the exact
unreliability the
continue-on-errorwas papering over — so the scan can gatefor real. Comments/summary were already disabled via env vars, so the action
provided only the scan; the swap loses no functionality and drops the
now-unneeded
GITHUB_TOKENenv andfetch-depth: 0. (secrets: inheritis nolonger required by this job; harmless if callers keep it.)
Verification
Exercised the exact install + scan logic locally with the pinned binary:
gitleaks_8.18.4_linux_x64.tar.gzmatches (sha256sum -c→ OK);actionlintclean on the changed steps.Note: reusable-workflow changes are not self-tested in this repo — the caller
secret-scanner.ymlpins the reusable at a fixed SHA, so this repo's own PR CIruns the old reusable. The change is exercised once a consumer re-pins to a SHA
containing it.
Running the new gate against this repo's own tree surfaces 15 findings —
all false positives, not real secrets:
avow-protocol/**/deno.lock(integrity hashes).md/examples.ndjsonlol/analysis/Project.toml(Julia UUIDs)docs/audits/…a2mlavow-lib/ffi/zig/src/example.zigLockfiles and UUID-bearing manifests are everywhere in the estate, so flipping
gitleaks to gating without a shared allowlist would cry wolf across many
repos — noise that trains people to ignore the gate, the opposite of the goal.
Companion work needed before re-pinning consumers: a canonical
.gitleaks.toml(allowlist*.lock, UUID manifests,example/fixture/docspaths) passed to callers via
--config, or per-repo allowlists. Kept out of thisPR to keep it surgical.
Related follow-ups (separate PRs)
governance-reusable.yml: "Code quality + docs" and "Guix primary / Nixfallback policy" jobs currently cannot fail (no
exit 1); the redundanttrufflehog there (also
continue-on-error) is better removed per thisreusable's own "trufflehog removed as redundant" note. Deferred pending a
decision on the existing
advisory-firstdesign stance.