Skip to content

Make safe-output run summary entries regular and always linked - #51478

Merged
dsyme merged 5 commits into
mainfrom
copilot/fix-safe-outputs-summary-links
Aug 9, 2026
Merged

Make safe-output run summary entries regular and always linked#51478
dsyme merged 5 commits into
mainfrom
copilot/fix-safe-outputs-summary-links

Conversation

Copilot AI commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

The safe-output section of the workflow run summary rendered inconsistently: entries whose handlers report entity fields under non-canonical names (e.g. close_pull_requestpull_request_url, reply_to_pull_request_review_commentreply_url) showed nothing but a heading, labels rendered as [object Object], and the section was a bare ## heading rather than a collapsible block like the threat-detection sections.

All changes are in actions/setup/js/safe_output_summary.cjs.

Uniform entity link

  • Replaced the ad-hoc **URL:** / **Location:** / **Project URL:** lines with a single **Target:** line.
  • The URL, number, and repo are resolved from priority-ordered field lists (url, html_url, issue_url, pull_request_url, discussion_url, comment_url, reply_url, review_url, item_url, commit_url, projectUrl, run_url, …), so every handler shape produces a link without each handler having to normalize its result.
  • When a handler reports only repo + number (e.g. add_labels), the link is derived as <GITHUB_SERVER_URL>/owner/repo/issues/<n>, which GitHub redirects to the PR when the number is a PR.
  • Only http(s) URLs are linkified; anything else degrades to plain text.
  • The fallback issue/PR branches use the same link formatting.

Labels

  • Labels are normalized from strings or GitHub label objects ({name}), preferring handler-reported labelsAdded/labels over the raw message labels. Fixes **Labels:** [object Object].

Section framing

  • The whole section is now wrapped in <details> with a status emoji and counts in the <summary>, matching the threat-detection sections. The Processed N … / Status: / item-count lines are unchanged, just moved inside.

Before / after for the same run:

-## Safe Output Processing Summary
+<details>
+<summary>✅ Safe Output Processing Summary (5 succeeded, 0 failed)</summary>

 <details>
 <summary>✅ Close Pull Request - Success (Message 1)</summary>

 ### Close Pull Request
+
+**Target:** [#445738](https://github.com/github/github/pull/445738)

 </details>

 <details>
 <summary>✅ Add Labels - Success (Message 7)</summary>

 ### Add Labels

-**Location:** github/github#445502
-
-**Labels:** [object Object]
+**Target:** [github/github#445502](https://github.com/github/github/issues/445502)
+
+**Labels:** bug, chore

 </details>
+
+</details>

Tests in actions/setup/js/safe_output_summary.test.cjs cover close-PR links, review-comment reply links, label-object normalization, derived entity URLs, non-http URL handling, and the collapsible wrapper.

Co-authored-by: dsyme <7204669+dsyme@users.noreply.github.com>
Copilot AI changed the title [WIP] Improve safe outputs summary to include links and regularity Make safe-output run summary entries regular and always linked Aug 9, 2026
Copilot AI requested a review from dsyme August 9, 2026 00:03
@dsyme
dsyme marked this pull request as ready for review August 9, 2026 00:09
Copilot AI balanced review requested due to automatic review settings August 9, 2026 00:09
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

PR Code Quality Reviewer completed the code quality review.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • api.individual.githubcopilot.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.individual.githubcopilot.com"

See Network Configuration for more information.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Test Quality Sentinel completed test quality analysis.

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Ponytail Reviewer completed successfully!

Generated by Ponytail Reviewer for #51478

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR does not have the 'implementation' label and has 0 new lines of code in business logic directories (threshold: 100).

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Regularizes safe-output summaries with consistent target links, label formatting, and collapsible framing.

Changes:

  • Adds target URL discovery and derived issue/PR links.
  • Normalizes label objects into readable names.
  • Adds collapsible summary framing and coverage for key result shapes.
Show a summary per file
File Description
actions/setup/js/safe_output_summary.cjs Implements target formatting, label normalization, and collapsible summaries.
actions/setup/js/safe_output_summary.test.cjs Tests links, labels, URL safety, and framing.

Review details

Tip

Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 2/2 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread actions/setup/js/safe_output_summary.cjs Outdated
Comment thread actions/setup/js/safe_output_summary.cjs Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes are clean and well-tested.

  • Entity URL extraction is centralized via priority-ordered field lists
  • XSS is guarded via the https?:// regex in formatLink — non-http URLs render as plain text (tested)
  • Graceful fallback: buildEntityUrl constructs links from repo+number when no URL is reported
  • <details> wrapping is consistent with other summary sections
  • Label normalization correctly handles both plain strings and GitHub label objects

No blocking issues.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 21.6 AIC · ⌖ 8.94 AIC · ⊞ 5.5K

@dsyme

dsyme commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

@copilot Address the above review comments

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generated by ✂️ Ponytail Reviewer for #51478 · auto · 41.3 AIC · ⌖ 4.76 AIC · ⊞ 6.8K
Comment /ponytail to run again

Comment thread actions/setup/js/safe_output_summary.cjs Outdated
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Test Quality Analysis 🧪

PR #51478: "Make safe-output run summary entries regular and always linked"

Summary

This PR adds 7 new behavioral tests to actions/setup/js/safe_output_summary.test.cjs, all validating the new entity-link formatting feature. The tests achieve excellent coverage of design invariants, edge cases, and security boundaries.

Key Metrics

  • Test Quality Score: 91/100 ✅ Excellent
  • Design Tests: 7/7 (100%)
  • Tests with Edge Cases/Boundaries: 5/7 (71%)
  • Test Inflation Ratio: 0.67:1 (100 test lines ÷ 150 production lines) — well below 2:1 threshold
  • Implementation Tests: 0% (target: ≤ 30%) ✅
  • Hard Violations: None ✅

Test Coverage Breakdown

Detailed Test Classifications
Test Name Design Value Type Assertions Edge Cases
link to closed PR ✅ Contract High Design 1
link to review comment ✅ Contract High Design 2
render label objects by name ✅ Contract High Design 2 Objects serialization
prefer result labels over message ✅ Contract High Design 2 Precedence resolution
derive entity link from repo/number ✅ Contract High Design 1 URL fallback generation
render plain text for non-http URLs ✅ Contract High Design 2 🔒 Security: XSS prevention
collapsible details structure ✅ Contract High Design 2 Structure validation

Strengths ✅

  1. 100% Behavioral Contracts — Every test verifies a design invariant (entity-link formatting consistency). Zero implementation-detail tests.

  2. Excellent Edge-Case Coverage — 71% of tests (5/7) explicitly validate boundary conditions:

    • Label object serialization (prevents "[object Object]" display bugs)
    • Label source precedence (message vs. result)
    • URL derivation from separate fields (when URL field missing)
    • Protocol validation (blocks `(redacted) URLs — security boundary)
    • HTML structure (collapsible rendering)
  3. Security-Focused Test — Test 6 validates that malicious protocol URLs (`(redacted) are rendered as plain text, preventing XSS injection in markdown. Excellent choice.

  4. Healthy Test Inflation Ratio — 0.67:1 is clean: tests are focused and proportional to the production code change, not inflated.

  5. Clear Test Isolation — Each test is independent with minimal setup, making failures easy to diagnose.

Notes

  • Tests focus on "happy path" success cases (appropriate for a formatting module; error handling is upstream).
  • No mock-library violations (minimal mocking, only for integration check in test 7).
  • One test assertion was modified to align with the new regular "Target:" field name (reflects design change correctly).

Recommendation

✅ APPROVE — This PR demonstrates strong test quality. All new tests are behavioral contracts covering the design's key invariants, including security boundaries. The security-focused protocol validation test is particularly commendable. Test inflation is healthy, and no coding violations detected.

🧪 Test quality analysis by Test Quality Sentinel · haiku45 · 22.3 AIC · ⌖ 3.09 AIC · ⊞ 7.7K ·
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 91/100. 0% implementation tests (threshold: 30%). All tests are behavioral contracts with excellent edge-case coverage, including security validation. No violations detected.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 57.4 AIC · ⌖ 7.77 AIC · ⊞ 7.1K
Comment /matt to run again

Comment thread actions/setup/js/safe_output_summary.cjs
Comment thread actions/setup/js/safe_output_summary.cjs Outdated
Comment thread actions/setup/js/safe_output_summary.cjs
Comment thread actions/setup/js/safe_output_summary.test.cjs
Co-authored-by: dsyme <7204669+dsyme@users.noreply.github.com>

Copilot AI commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Address the above review comments

Addressed in cb9ad00219: safe-output target rendering now falls back to message/context fields for sparse handler results, and tests cover the additional URL fields plus assign_milestone and update_project fallback cases.

Copilot AI requested a review from dsyme August 9, 2026 00:32
@dsyme
dsyme merged commit 0c1879e into main Aug 9, 2026
10 checks passed
@dsyme
dsyme deleted the copilot/fix-safe-outputs-summary-links branch August 9, 2026 00:33
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.86.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

In workflow run summary, each safe output should show links and be improved to be more regular

3 participants