Skip to content

[aw] Restore CI scanner repository lookup dependency - #135202

Merged
vitek-karas merged 3 commits into
dotnet:mainfrom
vitek-karas:vitek-karas-ci-scanner-verification
Oct 6, 2026
Merged

vitek-karas merged 3 commits into
dotnet:mainfrom
vitek-karas:vitek-karas-ci-scanner-verification

Conversation

@vitek-karas

Copy link
Copy Markdown
Member

Summary

Follow-up to #133684. The search wrapper worked, but the first post-merge scanner run stopped when reading candidate issues. The production gateway checks repository visibility using an internal search_repositories call, which was unavailable after removing the repos backend toolset:

unknown tool "search_repositories": Bad Request
  • Restore repos in the production GitHub backend toolsets and regenerate the lock file with its recorded compiler version, v0.86.2.
  • Keep the agent-facing tool list unchanged: issue_read, search_pull_requests, and pull_request_read.
  • Add regression checks for the repository lookup dependency and matching production/eval backend toolsets.
  • Document that the scanner eval does not exercise the production gateway.

Why earlier validation missed this

Live candidate-read testing did occur, but the eval connected directly to GitHub MCP with repos already enabled. Production uses a remote backend through the gateway. The deterministic tests covered the search wrapper and candidate-read grading, not the gateway's internal metadata calls, and no check compared the backend toolsets. Consequently, successful eval reads did not establish that the production configuration could read the same candidates.

Validation

  • node --test .github/workflows/evals/kbe-search.test.mjs: 26 passed, zero failed or skipped. Both new configuration checks failed against the original configuration before applying the fix.
  • gh-aw-v0.86.2 compile --schedule-seed dotnet/runtime --actionlint: all six workflows compiled. Actionlint reported existing diagnostics; the scanner diagnostics also occur in the original lock file and are unchanged by this patch.
  • gh-aw-v0.86.2 compile ci-failure-scan --schedule-seed dotnet/runtime: passed, preserving the scanner's existing failure-issue expiration setting.
  • Read-only smoke test through the production-pinned gateway v0.4.9: reproduced the missing internal lookup with the old toolsets; successfully read the same candidate issue after restoring repos. The gateway exposed exactly the same three agent tools in both configurations.
  • git diff --check: passed.

No full scheduled scanner run was dispatched. The separate model-selection and incomplete-run reporting problems are outside this change.

Note

These changes and this PR description were prepared with GitHub Copilot.

Keep the production GitHub backend toolsets aligned with the scanner eval and add focused regression checks for the gateway lookup dependency.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Preserve main's failure-reporting settings alongside the repository lookup dependency fix.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@vitek-karas
vitek-karas marked this pull request as ready for review October 5, 2026 12:34
@vitek-karas
vitek-karas requested review from a team and jeffhandley as code owners October 5, 2026 12:34
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
15 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@vitek-karas
vitek-karas merged commit a4dd108 into dotnet:main Oct 6, 2026
24 checks passed
@vitek-karas
vitek-karas deleted the vitek-karas-ci-scanner-verification branch October 6, 2026 07:24
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 12.0-preview1 milestone Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants