Skip to content

push_to_pull_request_branch samples with static 'branch:' fail in --use-samples mode (no fallback to current branch) #37546

Description

@dsyme

Summary

When a safe-outputs.push-to-pull-request-branch.samples[*] entry includes a static branch: "gh-aw-sample-...", sample replay (--use-samples) sends that value as tools/call arguments to the safe-outputs MCP server. The MCP handler then calls generateGitPatch(entry.branch, ...) against a checkout where the sample branch does not exist as a local ref, and returns:

Cannot generate incremental patch: refs/remotes/origin/gh-aw-sample-copilot-push-branch
is not present in checkout '/home/runner/work/gh-aw-test/gh-aw-test' and could not
be fetched (the safe-outputs MCP server has no credentials for private repositories).

No safe-output is written to outputs.jsonl. The downstream safe_outputs job logs Found 0 message(s) in agent output and the push-to-PR-branch flow is never exercised.

The post-agent override at push_to_pull_request_branch.cjs:470 (branchName = pullRequest.head.ref) does not help, because the message is rejected at sample-replay time and never reaches that handler.

Reproduction

Sample workflow excerpt:

safe-outputs:
  push-to-pull-request-branch:
    samples:
      - branch: "gh-aw-sample-copilot-push-branch"   # <-- sample-only branch
        message: "Multi-commit test push from Copilot"
        patch: |
          diff --git a/README-copilot-test.md b/README-copilot-test.md
          ...

Run with --use-samples (no AI engine). The MCP server fails to find refs/remotes/origin/gh-aw-sample-copilot-push-branch and emits an error response.

Example: https://github.com/githubnext/gh-aw-test/actions/runs/27091543149

Workaround in test code

Omit branch: from the sample entry. The MCP server then falls through to getCurrentBranch(repoCwd) (safe_outputs_handlers.cjs:924), which works during sample replay when the workspace is checked out on the PR's head branch (slash-command flow handles this via #37187).

Suggested fixes

Either of the following would remove the foot-gun:

  1. Ignore branch: in sample replay mode. When GH_AW_SAMPLES is set, treat branch: as advisory and always call getCurrentBranch(repoCwd). Sample-replay never has access to the user's eventual PR head branch, so a static value can only be wrong.
  2. Fall back to getCurrentBranch(repoCwd) when the sample branch can't be fetched. Today the handler hard-fails; demoting that to a warning + fallback would make samples self-healing.
  3. Document in safe-outputs.push-to-pull-request-branch.samples schema/docs that branch: should be omitted in samples for push-to-pull-request-branch. This is the least intrusive change.

Affected tests in githubnext/gh-aw-test

  • test-copilot-push-to-pull-request-branch
  • test-copilot-nosandbox-push-to-pull-request-branch
  • test-claude-push-to-pull-request-branch
  • test-codex-push-to-pull-request-branch

All four now use a sample without branch: to work around this, but the original (#37532-style) intent — samples that mirror what the AI agent would call — is lost.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions