[smoke-detector] 🔍 Smoke Test Investigation - Smoke Copilot: Permission Denied for Safe-Outputs Tools

[github-actions]

🔍 Smoke Test Investigation - Run #18779136174

Summary

The Smoke Copilot workflow failed because the Copilot agent received "Permission denied" when attempting to use safe-outputs MCP tools, despite the MCP server being properly configured and tools being correctly registered. This is a NEW failure pattern distinct from previous config parsing issues (#2280, #18778382550).

Failure Details

• Run: #18779136174
• Commit: ea550eb
• Branch: copilot/update-copilot-agent-engine
• Trigger: workflow_dispatch
• Duration: 3.0 minutes
• Failed Jobs: create_issue (3s duration)

Root Cause Analysis

Primary Error

From /tmp/gh-aw/aw-mcp/logs/run-18779136174/agent-stdio.log/agent-stdio.log:19:

✗ create_issue
   Permission denied and could not request permission from user

Investigation Findings

1. Safe-Outputs MCP Server Status: ✅ WORKING CORRECTLY

From the session logs, the safe-outputs MCP server:

• ✅ Started successfully
• ✅ Parsed config correctly: {"create_issue":{"max":1,"min":1},"missing_tool":{}}
• ✅ Registered tools: create_issue, missing_tool
• ✅ Responded to tool list requests

[safe-outputs-mcp-server] Successfully parsed config from environment
[safe-outputs-mcp-server] Final processed config: {"create_issue":{"max":1,"min":1},"missing_tool":{}}
[safe-outputs-mcp-server] v1.0.0 ready on stdio
[safe-outputs-mcp-server]   tools: create_issue, missing_tool

2. Agent Attempted Tool Usage: ✅ FOUND THE TOOL

The agent:

• ✅ Found the safe-outputs-create_issue tool
• ✅ Attempted to call it with valid arguments
• ✅ Built a comprehensive PR summary
• ❌ Received "Permission denied" error

Tool call ID: toolu_vrtx_01PnCprs8eQKssUSBfPg4W86

3. Permission System Blocked the Call: ❌ ROOT CAUSE

The error message "Permission denied and could not request permission from user" indicates the Copilot CLI's tool authorization system blocked the tool call. This is NOT:

• A config parsing error (that was fixed in commit ea550eb)
• An MCP server crash or misconfiguration
• A tool registration issue

This IS:

• A permission/authorization issue in the Copilot CLI or model-side tool filtering
• A potential missing permission declaration in the workflow
• A possible tool allowlist configuration problem

4. Downstream Impact

The create_issue job failed with:

Error reading agent output file: ENOENT: no such file or directory, 
open '/tmp/gh-aw/safe-outputs/agent_output.json'

This is expected since the agent couldn't use safe-outputs tools due to permission denial.

Failed Jobs and Errors

Job Sequence

1. ✅ activation - succeeded (2s)
2. ✅ agent - succeeded (1.1m) - BUT couldn't use safe-outputs tools
3. ✅ detection - succeeded (32s)
4. ❌ create_issue - failed (3s)
5. ⏭️ missing_tool - skipped

Error Summary

From audit report:

• Total Errors: 10
• Total Warnings: 4

Key Error:

• Permission denied when calling safe-outputs-create_issue tool

Comparison with Previous Issues

Issue	Error	Status	This Issue?
#2280	Malformed JSON - MCP server crashed	Closed (Fixed by ea550eb)	❌ Different
#18778382550	Config treated as character array	-	❌ Different

This is a NEW pattern: COPILOT_SAFE_OUTPUTS_PERMISSION_DENIED

Key Differences

Previous Issues (Config Parsing):

• MCP server failed to start or parse config
• Tools were not registered
• Agent never saw the tools

This Issue (Permission Denied):

• ✅ MCP server running correctly
• ✅ Tools registered properly
• ✅ Agent found the tools
• ❌ Permission system blocked tool usage

Commit Analysis

Commit ea550eb: "Fix double escaping of safe outputs config in MCP env vars"

This commit successfully FIXED the config parsing issue from #2280 by removing %q formatting. The MCP server now parses the config correctly, proving the fix worked.

However, this revealed a NEW underlying issue: the Copilot CLI's permission system is blocking safe-outputs tool usage.

Recommended Actions

Critical Priority ⚠️

• Investigate Copilot CLI tool permission configuration

  • Check if safe-outputs MCP tools need explicit permission grants
  • Review Copilot CLI 0.0.349 tool authorization behavior
  • Why: This is the direct cause of the permission denial

• Check workflow frontmatter for tool permissions

  # In .github/workflows/smoke-copilot.md frontmatter:
  tools:
    # Are safe-outputs tools explicitly allowed?

  • Compare with other workflows that successfully use safe-outputs
  • Why: Tools may need explicit declaration in workflow

• Test safe-outputs tools in isolation

  # Minimal test workflow with just safe-outputs MCP
  # Verify if permission issue is specific to this workflow or systemic

  • Why: Determine if issue is workflow-specific or engine-wide

High Priority

• Review Copilot CLI version compatibility

  • Agent version: 0.0.349
  • Check if this version has known issues with MCP tool permissions
  • Why: May be a regression or incompatibility

• Add debug logging for tool permissions

  • Log which tools are allowed/denied during agent execution
  • Capture full permission denial context
  • Why: Better diagnostics for future failures

• Check MCP tool allowlist configuration

  • Verify if Copilot engine has a tool allowlist separate from MCP registration
  • Compare with Claude engine implementation
  • Why: May need explicit allowlist entry for safe-outputs

Medium Priority

• Compare with other engines

  • Check if Claude engine has same issue with safe-outputs
  • Review how different engines handle MCP tool permissions
  • Why: Understand if this is Copilot-specific

• Add pre-flight permission check

  - name: Verify Safe Outputs Tools Available
    run: |
      # Check if safe-outputs tools are accessible before agent runs
      # Log permission status

  • Why: Early detection of permission issues

• Review MCP server capabilities negotiation

  • Check if capabilities are properly communicated during MCP handshake
  • Verify protocol version compatibility
  • Why: Permission issues could stem from capability negotiation

Prevention Strategies

1. Explicit Permission Declaration

   • Declare safe-outputs tools in workflow frontmatter if supported
   • Document permission requirements for each MCP server

2. Permission Verification

   • Add automated checks for tool permissions before agent execution
   • Log available tools and their permission status

3. Engine-Specific Configuration

   • Document Copilot CLI-specific permission requirements
   • Create engine-specific MCP configuration if needed

4. Better Error Messages

   • Request more detailed permission denial messages from Copilot CLI
   • Include tool name, permission type, and reason in error output

Technical Details

Environment Context

• Copilot CLI: 0.0.349
• Node.js: v24.10.0
• MCP Protocol: 2025-06-18
• Safe-Outputs MCP: v1.0.0
• Staged Mode: true

MCP Tool Registration

From session log:

{
  "tools": [
    {
      "name": "create_issue",
      "description": "Create a new GitHub issue",
      "inputSchema": {
        "type": "object",
        "required": ["title", "body"],
        "properties": {
          "title": {"type": "string"},
          "body": {"type": "string"},
          "labels": {"type": "array", "items": {"type": "string"}}
        }
      }
    },
    {
      "name": "missing_tool",
      "description": "Report a missing tool or functionality",
      "inputSchema": { ... }
    }
  ]
}

Tools were correctly registered with the Copilot CLI as safe-outputs-create_issue and safe-outputs-missing_tool.

Agent Tool Call Attempt

The agent attempted to call:

{
  "name": "safe-outputs-create_issue",
  "arguments": {
    "title": "Summary: Last 5 Merged Pull Requests",
    "body": "## Summary of Recent Merged Pull Requests\n\n[... comprehensive PR summary ...]"
  }
}

Arguments were valid and properly formatted.

Historical Context

Similar Permission Issues:

• [q] Fix Tidy workflow permissions - add missing edit tool #1738: Tidy workflow had permission denied for edit: tool (fixed by adding tool declaration)

Pattern Evolution: This is the first permission denial for safe-outputs MCP tools in the smoke tests. Previous safe-outputs failures were all config-related.

Success Sequence:

1. ✅ Fixed base64 config issue
2. ✅ Fixed malformed JSON config issue ([smoke-detector] 🔍 Smoke Test Investigation - Smoke Copilot: Safe-Outputs MCP Crashes Due to Malformed Config JSON #2280)
3. ✅ Fixed config-as-character-array issue
4. ❌ NEW: Permission denial when using tools

Related Information

• Branch: copilot/update-copilot-agent-engine (active development)
• Workflow Source: .github/workflows/smoke-copilot.md
• MCP Server: /tmp/gh-aw/safe-outputs/mcp-server.cjs
• Related Issues: [smoke-detector] 🔍 Smoke Test Investigation - Smoke Copilot: Safe-Outputs MCP Crashes Due to Malformed Config JSON #2280 (fixed), #18778382550 (related pattern)
• Related PR: Pass MCP config as CLI argument instead of file for Claude and Copilot engines #2264 (MCP config as CLI argument changes)

---

Investigation Metadata:

• Investigator: Smoke Detector
• Investigation Run: #18779212762
• Pattern ID: COPILOT_SAFE_OUTPUTS_PERMISSION_DENIED
• Severity: High
• Is Flaky: No
• Category: Permission/Authorization Error
• Investigation Saved: /tmp/gh-aw/cache-memory/investigations/2025-10-24-18779136174.json

Labels: smoke-test, investigation, copilot, safe-outputs, permission, high-priority, mcp

AI generated by Smoke Detector - Smoke Test Failure Investigator