You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This run reviewed 5 files in rotation slots 14-18: specs/security-architecture-spec-validation.md, specs/security-architecture-spec.md, scratchpad/github-mcp-access-control-specification.md, scratchpad/guard-policies-specification.md, and scratchpad/safe-outputs-specification.md (deprecated). The rotation has now completed a full pass over all 19 tracked spec files (last_index: 18 of 19) and will wrap to index 0 next run. The most significant finding is an overdue revalidation commitment: the security architecture spec's own Sync Notes promised a targeted §9/§11 revalidation by 2026-09-01, which has now passed (today is 2026-09-09) with no recorded completion. Appendix G.10 also documents six formal-test coverage gaps (G.1, G.2 partial, G.3 partial, G.4 partial, G.5 partial, G.8, G.9) that remain open. The deprecated scratchpad/safe-outputs-specification.md removal checklist is on track (deletion due 2026-09-21, zero stale references found today) but has no automated re-check scheduled before the deadline.
Priority Work Queue
P0: Missed §9/§11 revalidation deadline (2026-09-01) in specs/security-architecture-spec.md Sync Notes — no evidence a revalidation pass occurred; risk of undetected spec/implementation drift on runtime sequencing requirements.
P0: Appendix G.10 formal-test gaps for G.1 (action pinning) and G.8 (concurrency control) have zero dedicated tests — these are compile-time/structural guarantees with no regression protection.
P1: Appendix G.10 partial-coverage gaps for G.2 (safe_outputs job permissions), G.3 (fork protection if: assertion), G.4 (sanitized-prompt consumption), G.5 (detection job permissions: {} + gate) — existing tests cover adjacent behavior but not the literal checklist item.
P1: §12 Compliance Test Matrix gaps for T-PM-003/005/007, T-TD-002–007, and T-SI (sandbox host-visibility probe, tracked in #48686 but still open) — validation report marks these "lacks dedicated evidence" or "partially evidenced."
P2: scratchpad/safe-outputs-specification.md deletion checklist (deadline 2026-09-21) has no scheduled automated re-verification step before the deadline; a rotation pass should re-run the reference grep close to the date.
P2: Companion spec cross-reference (scratchpad/github-mcp-access-control-specification.md, scratchpad/guard-policies-specification.md) T-Tweaks to docs, gh aw help and notice #47–060 tests remain "not directly evidenced" in the top-level validation report per its own gap table.
SPDD Checklist
/spdd-sync — In specs/security-architecture-spec.md Sync Notes, either perform the promised §9/§11 revalidation pass against pkg/workflow/threat_detection_config.go and pkg/workflow/threat_detection_inline_engine.go and record results in specs/security-architecture-spec-validation.md, or update the Sync Notes with a new committed date if maintainer bandwidth requires deferral. Done when: specs/security-architecture-spec-validation.md has a dated entry referencing this pass and specs/security-architecture-spec.md Sync Notes no longer show a past-due date.
/spdd-generate — Add TestFormalG01_ActionsSHAPinned (or similarly named) to pkg/workflow/security_architecture_sg_formal_test.go that compiles a representative workflow and asserts every non-local uses: line matches owner/action@[0-9a-f]{40}. Done when: test exists, passes, and Appendix G.10's G.1 row in specs/security-architecture-spec.md is updated from gap to covered.
/spdd-generate — Add TestFormalG08_ConcurrencyGroupDynamic to pkg/workflow/security_architecture_sg_formal_test.go that compiles a PR-triggered workflow and asserts concurrency.group contains a dynamic expression (e.g., PR number) and cancel-in-progress matches the documented PR/issue convention. Done when: test exists, passes, and Appendix G.10's G.8 row is updated to covered.
/spdd-generate — Extend TestFormalStaged_HandlerRequiresNoWritePerms (or add a new test) in pkg/workflow/security_architecture_sg_formal_test.go to compile a safe-outputs-enabled workflow and assert the compiled safe_outputs job's permissions: block contains only the operations configured in frontmatter. Done when: G.2 safe_outputs row in Appendix G.10 moves from partial to covered.
/spdd-generate — Add a formal test asserting the compiled activation job's if: condition contains the github.event.pull_request.head.repo.id == github.repository_id fork-protection guard for pull_request-triggered workflows. Done when: G.3 row in Appendix G.10 moves from partial to covered.
/spdd-generate — Add a formal test asserting the compiled detection job has permissions: {} and that the compiled safe_outputs job's if: condition includes needs.detection.outputs.success == 'true'. Done when: G.5 row in Appendix G.10 moves from partial to covered.
/spdd-sync — Update specs/security-architecture-spec-validation.md §12 Compliance Test Matrix rows for T-PM-003/005/007 and T-TD-002–007 with either new evidence citations or an explicit tracking-issue reference, matching the pattern already used for T-SI (issue #48686). Done when: no row in the matrix says "lacks dedicated evidence entries" without a linked tracking issue.
/spdd-sync — In specs/safe-outputs-scratchpad-removal.md, add a dated checkbox for a re-run of grep -r "scratchpad/safe-outputs-specification.md" docs/ .github/ no later than 2026-09-18 (3 days before the 2026-09-21 deletion deadline) so the rotation can catch late-added references before deletion. Done when: checklist file has an explicit near-deadline re-verification item.
/spdd-analysis — Review scratchpad/github-mcp-access-control-specification.md §9.7 Resolved Decisions item about the --strict compile-time guard-policy dry-run report (deferred design question) and determine whether it should be promoted to a tracked follow-up or formally closed as out-of-scope. Done when: the open question either has a linked issue or an explicit "will not implement" note with rationale.
Analysis: This is the normative cross-reference of specs/security-architecture-spec.md against compiled .lock.yml output, last full pass 2026-07-15. Overall grade A, but its own §12 gap analysis lists Sandbox Isolation and Threat Detection test categories as only PARTIALLY EVIDENCED, and the Companion MCP Access-Control category (T-GH-47–060) as PARTIALLY EVIDENCED with no direct evidence in this document (deferred entirely to companion specs).
REASONS Canvas:
Requirements: Re-validation triggers are well-specified (normative + a documented decision log for trigger #3), but no automated mechanism enforces the 2026-09-01 targeted §9/§11 revalidation commitment made in the primary spec — this document does not yet reflect that pass having occurred.
Operations: The "Automation Approach" section proposes CI mechanisms (compiled-YAML structural assertions) that are not yet implemented; this is essentially the same work as the Appendix G.10 test gaps in the primary spec.
Safeguards: Failure Escalation procedure (grade downgrade + tracking issue) is well defined, but is not exercised — there is no evidence any of the six open Appendix G.10 gaps have triggered this escalation path, even though a full year could pass without resolution.
Generate tasks: See SPDD Checklist items for G.1, G.8, T-PM/T-TD gap items above; they all target this spec's validation state directly.
Sync tasks: Any future revalidation pass must refresh the §12 Gap Summary table and the "Recommendations for Specification Enhancement" section per the doc's own procedure.
specs/security-architecture-spec.md (2107 lines)
Analysis: Primary W3C-style normative specification (v1.0.1, Candidate Recommendation) covering 8 security layers plus compilation/runtime enforcement. Well-structured with 8 appendices including a Lock File Validation Checklist (Appendix G) and its formal-test coverage audit (G.10). The Sync Notes section is the key finding: it commits to a targeted §9/§11 revalidation "by 2026-09-01" — a date that has now passed without any visible completion record in the companion validation document.
REASONS Canvas:
Requirements: MUST/SHALL language is consistently applied via RFC 2119 conventions; no ambiguity found in the core layer definitions.
Structure: Appendix G.10's own coverage table is exemplary self-documentation of gaps (rare to see a spec audit its own test coverage this precisely) — this pattern should be encouraged in companion specs.
Norms: The Sync Notes revalidation-trigger norm ("A conforming implementation MUST re-run this validation when...") is undermined by the fact that a self-imposed deadline was missed without an explicit renegotiation entry — norms lose credibility if commitments silently lapse.
Safeguards: G.1 (action pinning) and G.8 (concurrency control) are both compile-time-invariant safeguards with literally zero regression test coverage; a future refactor could silently break either with no CI signal — the "gap" label is honest but does not reduce risk until closed.
Generate tasks: The six formal-test checklist items above (G.1, G.2, G.3, G.4, G.5, G.8, G.9 partial/gap rows) map directly to test files already named in Appendix G.10, minimizing discovery cost for implementers.
Sync tasks: Update the Sync Notes "Last Revalidation" paragraph once the §9/§11 pass completes, and cross-reference the new pass date in specs/security-architecture-spec-validation.md.
Analysis: Draft (v1.1.0) specification for GitHub MCP Server access control extensions (repo scoping, role filtering, integrity levels). Large and detailed with a dedicated Compliance Testing section and Compliance Fixture Stubs. One open resolved-decisions question remains explicitly deferred: whether the --strict compile-time guard-policy dry-run report should surface effective lockdown/guard-policy precedence.
REASONS Canvas:
Approach: The layered "guard policy → MCP Gateway middleware → schema extension" architecture (§10) is clearly described with a configuration-loading flow, reducing ambiguity for implementers.
Norms: §9.7 Resolved Decisions leaves one question genuinely open (not resolved) despite living under a "Resolved Decisions" heading — this is a naming/organization inconsistency that could confuse future readers who assume everything under that heading is closed.
Safeguards: §9.6 Safeguards section exists and appears comprehensive at a glance; no gaps flagged in this pass beyond the deferred design question above.
Generate tasks: Promote or formally close the deferred --strict dry-run design question (SPDD Checklist item above).
Sync tasks: This spec's own Sync Follow-ups (read during this review) already enumerate detailed multi-file update procedures for GP-01–GP-11 changes — no additional sync gaps identified this pass.
Analysis: Draft (v0.1.0) proposal for an extensible guard-policies framework in the MCP Gateway, starting with GitHub-specific policies (allowed-repos, min-integrity). Well-organized with Entities, Conformance, and five explicit Safeguards (GP-S001–GP-S005).
REASONS Canvas:
Entities: GitHubReposScope, GitHubIntegrityLevel, and GitHubToolConfig are formally defined with a documented deprecation note for the legacy repos field alias.
Safeguards: GP-S001 (Empty Allowlist Prevention) through GP-S005 (Absent Policy is Not Permissive) form a coherent fail-secure safeguard set; no gaps found in this pass.
Sync: The Sync Follow-ups section is thorough, naming exact files (pkg/workflow/tools_validation_github.go, mcp_github_config.go, tools_types.go) and both schema copies (pkg/workflow/schemas/ and docs/public/schemas/) that must stay in lockstep — a strong pattern other specs in this repo should adopt.
Generate tasks: None new this pass — the repos field alias removal (see Sync Notes) is already tracked with exact file/line references and does not need a new task.
Sync tasks: The doc's own "Removal tracking" note for the legacy repos alias is a good candidate for a future rotation's dedicated review once removal work begins.
Analysis: This file carries an explicit deprecation banner (v1.1.0, stale, superseded by the canonical docs/src/content/docs/specs/safe-outputs-specification.md v1.28.3) and a hard deletion deadline of 2026-09-21, tracked via specs/safe-outputs-scratchpad-removal.md. A live grep for scratchpad/safe-outputs-specification.md across docs/ and .github/ returned zero matches today, meaning the removal checklist's first blocking condition is currently satisfied.
REASONS Canvas:
Norms: The deprecation banner is exemplary — it names the canonical replacement, the archival deletion date, and an extension-request process (open an issue with docs label, tag @gh-aw-team). This pattern should be the template for any future scratchpad deprecation.
Safeguards: The removal checklist (specs/safe-outputs-scratchpad-removal.md) requires a zero-reference grep before deletion, which is good, but has no scheduled date for re-running that grep before the 2026-09-21 deadline — if a reference is added between now and then, it could go unnoticed until deletion day.
Generate tasks: None beyond the near-deadline re-verification item in the SPDD Checklist above.
Sync tasks: Once deleted (on or after 2026-09-21), the rotation's file_list in rotation.json must be regenerated to drop this path, and specs/safe-outputs-scratchpad-removal.md itself should be closed/archived.
Sync Follow-ups
After the §9/§11 revalidation pass completes: update specs/security-architecture-spec-validation.md §12 matrix and specs/security-architecture-spec.md Sync Notes "Last Revalidation" date in the same PR.
After any Appendix G.10 test is added: update the corresponding row in specs/security-architecture-spec.md Appendix G.10 from gap/partial to covered, and re-run pkg/workflow/security_architecture_sg_formal_test.go in full.
After scratchpad/safe-outputs-specification.md is deleted (on/after 2026-09-21): remove its entry from this rotation's file_list in /tmp/gh-aw/cache-memory/spdd-daily/rotation.json and close specs/safe-outputs-scratchpad-removal.md's checklist.
After any GP-01–GP-11 change in scratchpad/guard-policies-specification.md: cross-check pkg/workflow/schemas/mcp-gateway-config.schema.json and docs/public/schemas/mcp-gateway-config.schema.json stay identical per that spec's own Sync Follow-ups.
Context
Files reviewed this run (rotation indices 14-18 of 19, 5 files): specs/security-architecture-spec-validation.md, specs/security-architecture-spec.md, scratchpad/github-mcp-access-control-specification.md, scratchpad/guard-policies-specification.md, scratchpad/safe-outputs-specification.md
Rotation state: last_index: 18, total_files: 19 — full pass complete; next run resumes at index 0 (specs/awf-config-sources-compliance/README.md)
Summary
This run reviewed 5 files in rotation slots 14-18:
specs/security-architecture-spec-validation.md,specs/security-architecture-spec.md,scratchpad/github-mcp-access-control-specification.md,scratchpad/guard-policies-specification.md, andscratchpad/safe-outputs-specification.md(deprecated). The rotation has now completed a full pass over all 19 tracked spec files (last_index: 18of 19) and will wrap to index 0 next run. The most significant finding is an overdue revalidation commitment: the security architecture spec's own Sync Notes promised a targeted §9/§11 revalidation by 2026-09-01, which has now passed (today is 2026-09-09) with no recorded completion. Appendix G.10 also documents six formal-test coverage gaps (G.1, G.2 partial, G.3 partial, G.4 partial, G.5 partial, G.8, G.9) that remain open. The deprecatedscratchpad/safe-outputs-specification.mdremoval checklist is on track (deletion due 2026-09-21, zero stale references found today) but has no automated re-check scheduled before the deadline.Priority Work Queue
specs/security-architecture-spec.mdSync Notes — no evidence a revalidation pass occurred; risk of undetected spec/implementation drift on runtime sequencing requirements.if:assertion), G.4 (sanitized-prompt consumption), G.5 (detection jobpermissions: {}+ gate) — existing tests cover adjacent behavior but not the literal checklist item.#48686but still open) — validation report marks these "lacks dedicated evidence" or "partially evidenced."scratchpad/safe-outputs-specification.mddeletion checklist (deadline 2026-09-21) has no scheduled automated re-verification step before the deadline; a rotation pass should re-run the reference grep close to the date.scratchpad/github-mcp-access-control-specification.md,scratchpad/guard-policies-specification.md) T-Tweaks to docs,gh aw helpand notice #47–060 tests remain "not directly evidenced" in the top-level validation report per its own gap table.SPDD Checklist
/spdd-sync— Inspecs/security-architecture-spec.mdSync Notes, either perform the promised §9/§11 revalidation pass againstpkg/workflow/threat_detection_config.goandpkg/workflow/threat_detection_inline_engine.goand record results inspecs/security-architecture-spec-validation.md, or update the Sync Notes with a new committed date if maintainer bandwidth requires deferral. Done when:specs/security-architecture-spec-validation.mdhas a dated entry referencing this pass andspecs/security-architecture-spec.mdSync Notes no longer show a past-due date./spdd-generate— AddTestFormalG01_ActionsSHAPinned(or similarly named) topkg/workflow/security_architecture_sg_formal_test.gothat compiles a representative workflow and asserts every non-localuses:line matchesowner/action@[0-9a-f]{40}. Done when: test exists, passes, and Appendix G.10's G.1 row inspecs/security-architecture-spec.mdis updated from gap to covered./spdd-generate— AddTestFormalG08_ConcurrencyGroupDynamictopkg/workflow/security_architecture_sg_formal_test.gothat compiles a PR-triggered workflow and assertsconcurrency.groupcontains a dynamic expression (e.g., PR number) andcancel-in-progressmatches the documented PR/issue convention. Done when: test exists, passes, and Appendix G.10's G.8 row is updated to covered./spdd-generate— ExtendTestFormalStaged_HandlerRequiresNoWritePerms(or add a new test) inpkg/workflow/security_architecture_sg_formal_test.goto compile a safe-outputs-enabled workflow and assert the compiledsafe_outputsjob'spermissions:block contains only the operations configured in frontmatter. Done when: G.2 safe_outputs row in Appendix G.10 moves from partial to covered./spdd-generate— Add a formal test asserting the compiledactivationjob'sif:condition contains thegithub.event.pull_request.head.repo.id == github.repository_idfork-protection guard forpull_request-triggered workflows. Done when: G.3 row in Appendix G.10 moves from partial to covered./spdd-generate— Add a formal test asserting the compileddetectionjob haspermissions: {}and that the compiledsafe_outputsjob'sif:condition includesneeds.detection.outputs.success == 'true'. Done when: G.5 row in Appendix G.10 moves from partial to covered./spdd-sync— Updatespecs/security-architecture-spec-validation.md§12 Compliance Test Matrix rows for T-PM-003/005/007 and T-TD-002–007 with either new evidence citations or an explicit tracking-issue reference, matching the pattern already used for T-SI (issue#48686). Done when: no row in the matrix says "lacks dedicated evidence entries" without a linked tracking issue./spdd-sync— Inspecs/safe-outputs-scratchpad-removal.md, add a dated checkbox for a re-run ofgrep -r "scratchpad/safe-outputs-specification.md" docs/ .github/no later than 2026-09-18 (3 days before the 2026-09-21 deletion deadline) so the rotation can catch late-added references before deletion. Done when: checklist file has an explicit near-deadline re-verification item./spdd-analysis— Reviewscratchpad/github-mcp-access-control-specification.md§9.7 Resolved Decisions item about the--strictcompile-time guard-policy dry-run report (deferred design question) and determine whether it should be promoted to a tracked follow-up or formally closed as out-of-scope. Done when: the open question either has a linked issue or an explicit "will not implement" note with rationale.Per-Spec Findings
specs/security-architecture-spec-validation.md(756 lines)Analysis: This is the normative cross-reference of
specs/security-architecture-spec.mdagainst compiled.lock.ymloutput, last full pass 2026-07-15. Overall grade A, but its own §12 gap analysis lists Sandbox Isolation and Threat Detection test categories as only PARTIALLY EVIDENCED, and the Companion MCP Access-Control category (T-GH-47–060) as PARTIALLY EVIDENCED with no direct evidence in this document (deferred entirely to companion specs).REASONS Canvas:
#3), but no automated mechanism enforces the 2026-09-01 targeted §9/§11 revalidation commitment made in the primary spec — this document does not yet reflect that pass having occurred.Generate tasks: See SPDD Checklist items for G.1, G.8, T-PM/T-TD gap items above; they all target this spec's validation state directly.
Sync tasks: Any future revalidation pass must refresh the §12 Gap Summary table and the "Recommendations for Specification Enhancement" section per the doc's own procedure.
specs/security-architecture-spec.md(2107 lines)Analysis: Primary W3C-style normative specification (v1.0.1, Candidate Recommendation) covering 8 security layers plus compilation/runtime enforcement. Well-structured with 8 appendices including a Lock File Validation Checklist (Appendix G) and its formal-test coverage audit (G.10). The Sync Notes section is the key finding: it commits to a targeted §9/§11 revalidation "by 2026-09-01" — a date that has now passed without any visible completion record in the companion validation document.
REASONS Canvas:
Generate tasks: The six formal-test checklist items above (G.1, G.2, G.3, G.4, G.5, G.8, G.9 partial/gap rows) map directly to test files already named in Appendix G.10, minimizing discovery cost for implementers.
Sync tasks: Update the Sync Notes "Last Revalidation" paragraph once the §9/§11 pass completes, and cross-reference the new pass date in
specs/security-architecture-spec-validation.md.scratchpad/github-mcp-access-control-specification.md(2881 lines)Analysis: Draft (v1.1.0) specification for GitHub MCP Server access control extensions (repo scoping, role filtering, integrity levels). Large and detailed with a dedicated Compliance Testing section and Compliance Fixture Stubs. One open resolved-decisions question remains explicitly deferred: whether the
--strictcompile-time guard-policy dry-run report should surface effective lockdown/guard-policy precedence.REASONS Canvas:
Generate tasks: Promote or formally close the deferred
--strictdry-run design question (SPDD Checklist item above).Sync tasks: This spec's own Sync Follow-ups (read during this review) already enumerate detailed multi-file update procedures for GP-01–GP-11 changes — no additional sync gaps identified this pass.
scratchpad/guard-policies-specification.md(642 lines)Analysis: Draft (v0.1.0) proposal for an extensible guard-policies framework in the MCP Gateway, starting with GitHub-specific policies (
allowed-repos,min-integrity). Well-organized with Entities, Conformance, and five explicit Safeguards (GP-S001–GP-S005).REASONS Canvas:
GitHubReposScope,GitHubIntegrityLevel, andGitHubToolConfigare formally defined with a documented deprecation note for the legacyreposfield alias.pkg/workflow/tools_validation_github.go,mcp_github_config.go,tools_types.go) and both schema copies (pkg/workflow/schemas/anddocs/public/schemas/) that must stay in lockstep — a strong pattern other specs in this repo should adopt.Generate tasks: None new this pass — the
reposfield alias removal (see Sync Notes) is already tracked with exact file/line references and does not need a new task.Sync tasks: The doc's own "Removal tracking" note for the legacy
reposalias is a good candidate for a future rotation's dedicated review once removal work begins.scratchpad/safe-outputs-specification.md(1324 lines, DEPRECATED)Analysis: This file carries an explicit deprecation banner (v1.1.0, stale, superseded by the canonical
docs/src/content/docs/specs/safe-outputs-specification.mdv1.28.3) and a hard deletion deadline of 2026-09-21, tracked viaspecs/safe-outputs-scratchpad-removal.md. A live grep forscratchpad/safe-outputs-specification.mdacrossdocs/and.github/returned zero matches today, meaning the removal checklist's first blocking condition is currently satisfied.REASONS Canvas:
docslabel, tag@gh-aw-team). This pattern should be the template for any future scratchpad deprecation.specs/safe-outputs-scratchpad-removal.md) requires a zero-reference grep before deletion, which is good, but has no scheduled date for re-running that grep before the 2026-09-21 deadline — if a reference is added between now and then, it could go unnoticed until deletion day.Generate tasks: None beyond the near-deadline re-verification item in the SPDD Checklist above.
Sync tasks: Once deleted (on or after 2026-09-21), the rotation's
file_listinrotation.jsonmust be regenerated to drop this path, andspecs/safe-outputs-scratchpad-removal.mditself should be closed/archived.Sync Follow-ups
specs/security-architecture-spec-validation.md§12 matrix andspecs/security-architecture-spec.mdSync Notes "Last Revalidation" date in the same PR.specs/security-architecture-spec.mdAppendix G.10 from gap/partial to covered, and re-runpkg/workflow/security_architecture_sg_formal_test.goin full.scratchpad/safe-outputs-specification.mdis deleted (on/after 2026-09-21): remove its entry from this rotation'sfile_listin/tmp/gh-aw/cache-memory/spdd-daily/rotation.jsonand closespecs/safe-outputs-scratchpad-removal.md's checklist.scratchpad/guard-policies-specification.md: cross-checkpkg/workflow/schemas/mcp-gateway-config.schema.jsonanddocs/public/schemas/mcp-gateway-config.schema.jsonstay identical per that spec's own Sync Follow-ups.Context
specs/security-architecture-spec-validation.md,specs/security-architecture-spec.md,scratchpad/github-mcp-access-control-specification.md,scratchpad/guard-policies-specification.md,scratchpad/safe-outputs-specification.mdlast_index: 18,total_files: 19— full pass complete; next run resumes at index 0 (specs/awf-config-sources-compliance/README.md)/tmp/gh-aw/cache-memory/spdd-daily/rotation.json