Skip to content

Regularize conclusion no-op summaries - #51487

Merged
dsyme merged 1 commit into
mainfrom
copilot/fix-conclusion-summary-links
Aug 9, 2026
Merged

Regularize conclusion no-op summaries#51487
dsyme merged 1 commit into
mainfrom
copilot/fix-conclusion-summary-links

Conversation

@dsyme

@dsyme dsyme commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Extends #51478's fix for #51477 to the summary emitted by the conclusion job for delegated noop safe outputs.

Changes

  • render conclusion no-op output as a collapsible summary with regular per-message entries
  • link the summary to the workflow run when an HTTP(S) run URL is available
  • share the formatter between the normal no-op handler and the status-comment fallback
  • redact credential-shaped content before writing the step summary

Validation

  • GH_AW_PROMPTS_DIR="$PWD/../md" npx vitest run --no-file-parallelism conclusion_summary.test.cjs handle_noop_message.test.cjs notify_comment_error.test.cjs (59 tests passed)
  • make fmt-cjs && make lint-cjs (passed)
  • make agent-report-progress passed formatting, build, schema freshness, lint, and Go checks; setup JavaScript tests remain blocked by 11 pre-existing TypeScript errors in unrelated files

Copilot AI balanced review requested due to automatic review settings August 9, 2026 00:50
@dsyme
dsyme marked this pull request as draft August 9, 2026 00:51
@dsyme
dsyme marked this pull request as ready for review August 9, 2026 00:51
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Test Quality Sentinel completed test quality analysis.

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Ponytail Reviewer completed successfully!

Ponytail review: reviewed conclusion_summary.cjs and its two call sites. Change extracts a small shared formatter used by 2 existing callers, straightforward validation logic (runUrl scheme check), no reinvented stdlib, no unneeded deps, no speculative abstraction. Lean already. Ship.

Generated by Ponytail Reviewer for #51487

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #51487 does not have the 'implementation' label and has 0 new lines of code in business logic directories (threshold: 100).

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

PR Code Quality Reviewer completed the code quality review.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • api.individual.githubcopilot.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.individual.githubcopilot.com"

See Network Configuration for more information.

🔎 Code quality review by PR Code Quality Reviewer

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Regularizes conclusion-job no-op summaries to match safe-output summary formatting.

Changes:

  • Adds a shared collapsible no-op summary formatter.
  • Links valid workflow run URLs and redacts credential-shaped content.
  • Updates both no-op handling paths with integration tests.
Show a summary per file
File Description
actions/setup/js/conclusion_summary.cjs Implements shared summary formatting and redaction.
actions/setup/js/conclusion_summary.test.cjs Tests formatting and URL handling.
actions/setup/js/handle_noop_message.cjs Uses the shared formatter.
actions/setup/js/handle_noop_message.test.cjs Verifies normal no-op summary output.
actions/setup/js/notify_comment_error.cjs Uses the formatter for fallback summaries.
actions/setup/js/notify_comment_error.test.cjs Tests the fallback summary path.

Review details

Tip

Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 6/6 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread actions/setup/js/conclusion_summary.test.cjs

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The refactoring cleanly extracts a shared buildNoopConclusionSummary helper and wires it into both handle_noop_message and notify_comment_error. The collapsible HTML structure, URL validation, and staged-mode flag are all handled correctly. Tests cover the key scenarios. No blocking issues found.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 19.7 AIC · ⌖ 7.13 AIC · ⊞ 5.5K

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /tdd and /codebase-design — approving with one minor suggestion.

📋 Key Themes & Highlights

Positive Highlights

  • ✅ Clean extraction of buildNoopConclusionSummary into a focused module — simple interface, consistent with existing patterns
  • ✅ URL validation (/^https?:\/\//) guards against `(redacted) injection in run links; tested explicitly
  • ✅ Tests cover the main happy path, staged mode, and the security-relevant non-http URL case
  • redactStepSummaryContent applied at the formatter boundary so callers don't have to remember it
  • handle_noop_message and notify_comment_error now share the same formatter — good DRY win

Minor Gap

  • One edge case not yet covered: buildNoopConclusionSummary([]) (empty array). See inline comment.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 22.5 AIC · ⌖ 7.52 AIC · ⊞ 7.1K
Comment /matt to run again

summary += staged ? "The following messages would be logged if staged mode was disabled:\n\n" : "The following messages were logged for transparency:\n\n";
summary += target;

for (let index = 0; index < messages.length; index++) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] No test covers an empty messages array — buildNoopConclusionSummary([]) produces "0 no-op messages" with an empty body. Worth adding a quick test to document the intended behaviour and guard against future regressions.

💡 Suggested test
it('handles empty messages array gracefully', () => {
  const summary = buildNoopConclusionSummary([]);
  expect(summary).toContain('0 no-op messages');
  expect(summary).not.toContain('No-Op - Success');
});

@copilot please address this.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel

PR: #51487 — "Regularize conclusion no-op summaries"
Test Quality Score: 68/100 ⚠️ Acceptable
Implementation Test Ratio: 0/4 (0% ✅ Pass)


Summary

This PR introduces a new module (conclusion_summary.cjs) with 3 well-structured tests and modifies 2 existing test files to integrate with the new no-op summary formatting. All tests verify behavioral contracts — user-visible output rendering and security invariants.

Key Quality Signals:

  • 100% design tests (all 4 tests verify observable behavior)
  • XSS prevention edge case covered (non-HTTP URL filtering)
  • Proper resource cleanup in async tests (temp file management)
  • Clear assertions with descriptive expected values
  • ⚠️ Test inflation flag on notify_comment_error.test.cjs (11:1 ratio) — justified by fallback-path coverage

Per-Test Classification

Click to expand test analysis
Test File Classification Value Assessment
renders regular no-op entries inside a linked conclusion summary conclusion_summary.test.cjs behavioral_contract high_value ✅ Verifies main output format and link rendering
renders staged no-op messages as a preview conclusion_summary.test.cjs behavioral_contract high_value ✅ Covers preview mode variant
does not link non-http run targets conclusion_summary.test.cjs behavioral_contract high_value ✅ Security-relevant: XSS prevention via URL scheme validation
writes conclusion summary for noop messages (in notify_comment_error) notify_comment_error.test.cjs behavioral_contract high_value ✅ Covers fallback path; includes proper temp file cleanup

Line Inflation Analysis:

  • conclusion_summary.test.cjs: 31 test lines / 34 source lines = 0.91:1
  • notify_comment_error.test.cjs: +22 test lines / +2 source lines = 11:1 ⚠️ (justifiable: sets up fallback scenario with temp I/O)
  • handle_noop_message.test.cjs: +3 lines (integrates with existing test)

Violations & Flags

No hard violations detected.

  • No Go mock libraries (N/A — JavaScript tests)
  • No missing build tags (N/A — JavaScript tests)
  • All assertions include descriptive expected values

Soft Flag (monitoring):

  • notify_comment_error.test.cjs has a 11:1 line inflation ratio, but this is driven by legitimate setup/teardown for temporary file I/O. Not a violation.

Recommendation

ACCEPTABLE. Tests provide solid behavioral coverage with proper edge-case handling and resource management. Implementation test ratio is 0% (all are design tests), well below the 30% threshold. Proceed to merge.

🧪 Test quality analysis by Test Quality Sentinel · haiku45 · 20.5 AIC · ⌖ 3.06 AIC · ⊞ 7.7K ·
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 68/100. 0% implementation tests (threshold: 30%). All 4 tests are behavioral contracts with edge-case coverage (XSS prevention, fallback paths). No hard violations.

@github-actions github-actions Bot mentioned this pull request Aug 9, 2026
@dsyme
dsyme merged commit bbee985 into main Aug 9, 2026
69 of 74 checks passed
@dsyme
dsyme deleted the copilot/fix-conclusion-summary-links branch August 9, 2026 01:04
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot Quick triage nudge for this PR.

Please refresh the branch if GitHub can update it cleanly, re-check the current maintainer-facing state, run the pr-finisher skill, and push follow-up fixes only if something actionable remains.

Open review context (newest first):

  • Minor review suggestion Re-check whether any maintainer-facing follow-up remains from the recent approved reviews, including the empty-array test suggestion, and confirm readiness for merge.

Run: https://github.com/github/gh-aw/actions/runs/31287357931

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 20.7 AIC · ⌖ 6.68 AIC · ⊞ 8.5K ·
Comment /souschef to run again

@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.86.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants