Repository navigation
fix(inputs): a codex_config source is one loaded source per MCP server - #442
Conversation
A `tool_sources[]` row of `type: codex_config` failed outright with `InputParseError` — "A tool source loader reported tool 't_codex' as belonging to a different source than the one it was read from" — whenever the workspace contained a `.mcp.json` or `.codex/config.toml` declaring a server. Since a server with no enumerable tools still mints a `<server>.*` wildcard, *any* server at all reached it: the source type worked only on workspaces that had no MCP declarations to read, which is the opposite of what it is for. `inputs.mcp_manifest` minted a per-server id for every tool (`mcp_json:<server>`, `codex_config_mcp:<server>`) and then wrapped a whole file's servers in one `LoadedToolSource` named for the *file* (`codex_config_mcp:<path>`). `core.tool_identity` requires a loaded source to name the id its tools carry — a tool arriving under another source's name is a loader defect — so the two spellings could never agree. The loader now emits one `LoadedToolSource` per MCP server, built from the server, so there is no second place that spells the id. The minted ids themselves are unchanged and stay deliberately path-free: an MCP capability is its server and tool, so moving a `.mcp.json` remains a pure rename rather than a capability addition (`mcp audit` pins that). One source per *file* was the other candidate repair and is wrong: `_native_locator` is the bare tool name for MCP-like sources, so grouping a file's servers together made a legal `.mcp.json` whose two servers both expose `query` fail with "defines the tool 'query' more than once". Why no test caught it: the three existing `type: codex_config` fixtures (`test_verify_orchestrator.py`, `test_preflight.py`, `test_codex_boundary_check.py`) all point at workspaces where the loader mints no tools, so nothing reached the check. The loader-contract failure also gains a way forward. "That is a defect in the loader, not in this repository's configuration" was accurate and terminal: the reader cannot edit an adapter they do not own, and had just been told their own configuration was not at fault. When the dispatcher can prove which `tool_sources` entry produced the read, the message now adds the workaround after the diagnosis, named as the workaround it is. `details` gains `configured_source_id`. No version moves and no schema changes. Nothing that previously produced a report changes shape, because no workspace with an MCP server could produce one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review follow-ups on the loaded-source-per-server fix. Two comments illustrated the codex minted-id shape as `mcp_json:<server>` alone. That is one of three: `.codex/config.toml` mints `codex_config_mcp:` and a plugin-nested server mints `codex_plugin_config_mcp:<plugin>:`. The `configured_source_id` comment is the reference for why declarations must not be joined on `source_id` (#410 review), so a reader reasoning from one prefix handles the minority shape and misses the two that Codex adopters actually produce. Both now name all three, and the domain comment no longer states "matches no configured row" as a guarantee: a short server name is something an adopter could plausibly reuse as a `tool_sources[].id`, which the old path-based spelling made near-impossible. Two tests claimed coverage they did not assert. `test_every_minted_codex_mcp_tool_names_the_source_it_was_read_from` named a plugin-nested server and both file kinds in its docstring but asserted only that no tool mismatched its source, which a loader that stopped emitting a whole branch satisfies vacuously. It now pins the exact minted source ids and their tools. `test_the_minted_server_id_stays_free_of_the_path_it_was_read_from` guarded the rejected file-qualification for `mcp_json` only, so a cross-file fix that qualified just the TOML id would regress `mcp audit`'s pure-rename invariant with every guard green. It is now parametrized over all three prefixes. Both perturbations confirmed to fail the intended tests: qualifying only the TOML id fails the two new parametrizations, and dropping the plugin branch fails the exact-id assertion. Neither was caught before this commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two remaining partial enumerations, for consistency with the ones corrected in the previous commit: the path-free rationale applies to a `.codex/config.toml` as much as to a `.mcp.json`, and the mismatch the per-server split fixes covered the plugin-nested prefix too. Comments only; no behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pengfei-threemoonslab
left a comment
There was a problem hiding this comment.
Expert review found four actionable issues: two catalog-completeness/identity blockers and two normalization/recovery defects. The per-server split fixes the reported happy path, but some accepted inputs can still disappear from the analyzed surface or be conflated under one provider.
Validation performed on this exact head merged with current main (fc96bad91):
- 367 relevant adapter/identity/binding/verification tests passed.
- Ruff and compileall passed on the touched files/tree.
- All current GitHub checks are green.
- Agents Shipgate verification receipt validated; release decision
passed, merge verdictmergeable, controlcomplete. Those static checks do not exercise the adversarial input shapes called out inline.
I recommend fixing the P1 findings before merge and covering all four repros with end-to-end tests.
| ) | ||
| ) | ||
| return [_loaded_source_from_servers(source_ref, enriched)] if enriched else [] | ||
| return _loaded_sources_from_servers(enriched) |
There was a problem hiding this comment.
[P1] Preserve rejected MCP servers as typed omissions
_servers_from_mapping() silently skips every non-object server value, and this new return publishes only the normalized rows. A mixed input such as {"mcpServers":{"good":{"command":"node","tools":{"read_docs":{}}},"dropped":"not-an-object"}} now completes run_scan with good in tool_catalog, source_warnings=[], and no SourceSurfaceOmission for dropped. Before this PR the valid row hit the loader mismatch and aborted; after this change the affected config can reach a report with part of its declared surface missing. Please reject malformed server rows or attach a warning plus typed omission/evidence gap, following inputs/mcp.py, for JSON, top-level TOML, and plugin-nested maps. Add mixed-valid/invalid end-to-end tests.
| # ``.mcp.json`` or a ``.codex/config.toml`` — is not a capability | ||
| # change (``mcp audit`` pins that as a pure rename). The file it was | ||
| # read from travels on ``source_ref``/``source_path`` instead. | ||
| source_id = f"{source_id_prefix}:{server_name}" |
There was a problem hiding this comment.
[P1] Fail closed when a path-free provider ID is not unique
The path-free key is not unique among simultaneous declarations. Two nested .mcp.json files that both declare server github with disjoint tool names produce two sources named mcp_json:github; _observations accepts both because their native locators differ, silently assigns both tools one provider, and emits no warning. Only overlapping tool names hit the documented failure, so the changelog's claim that duplicate server declarations always fail is incomplete. The raw : composition also collides for plugin a:b / server c versus plugin a / server b:c. Track the origin tuple and fail closed when one minted ID comes from multiple origins, or use an injective component encoding while keeping relocation equivalence separate. Please add disjoint cross-file and delimiter-collision tests.
|
|
||
| return [ | ||
| LoadedToolSource( | ||
| source_id=server.source_id, |
There was a problem hiding this comment.
[P2] Canonicalize server IDs before enforcing source equality
A server key ending in whitespace still triggers the exact loader defect this PR is meant to remove. With .mcp.json containing {"mcpServers":{"srv ":{"command":"node","tools":{"query":{}}}}}, this row preserves mcp_json:srv while _observations strips the loaded ID to mcp_json:srv before comparing it with the unstripped tool ID, so run_scan aborts. Normalize or reject outer whitespace once when building the server identity (while preserving the raw key for source pointers), apply the rule consistently to JSON/TOML/plugin forms, and add end-to-end regressions.
| configured = (loaded.configured_source_id or "").strip() | ||
| workaround = ( | ||
| " Until it is fixed, removing the tool_sources entry " | ||
| f"{configured!r} from shipgate.yaml lets the scan run " |
There was a problem hiding this comment.
[P2] Route loader defects without recommending source deletion
This workaround removes the only configured source, so the next scan omits that capability entirely. Even though the sentence discloses the cost, deletion is coverage-bypass guidance from a merge gate. It also does not satisfy the agent-mode recovery contract: the details have no typed failure, so input_parse_recovery() returns the generic review action with no path, while this prose hardcodes shipgate.yaml even when the actual manifest is nested or named differently. Keep the hard failure, add a stable mismatch discriminator and a recovery branch that uses details.manifest_path, and route to fixing/upgrading the adapter or explicit human review. If exclusion is intentional, require a reviewed, verifier-gated manifest change rather than presenting deletion as scanner recovery.
…1556bc # Conflicts: # CHANGELOG.md
…1556bc # Conflicts: # src/agents_shipgate/cli/scan/source_loading.py # src/agents_shipgate/core/domain.py # src/agents_shipgate/inputs/mcp_manifest.py # tests/test_mcp_manifest.py
pengfei-threemoonslab
left a comment
There was a problem hiding this comment.
Second-pass review of current head d0de953 after merging current main (b867f31, #445).
Disposition: changes still needed before merge. This review is posted by the PR author account, so GitHub permits only COMMENTED, but the remaining P2 is actionable.
- P2 — route loader defects without recommending source deletion remains PR-specific. A mismatching adapter now tells the user to remove its
tool_sourcesrow, so the next scan omits that capability surface. The exception still has no typedfailure; agent-mode recovery falls through to an unrelated generic action with no path, while the prose hardcodesshipgate.yamland points atreport.jsoneven though this abort can occur before that report is created. Keep the hard failure, add a stable mismatch discriminator plus manifest-aware recovery, and route to adapter repair or upgrade, or explicit human review. Do not present deleting the gated source as scanner recovery.
Status of the earlier threads after the merge:
- #445 now supplies the primary per-server loader fix and conservative same-server reconciliation across files read by one
codex_configrow. - Malformed server omission, outer-whitespace IDs, and the remaining cross-config or delimiter identity gaps still reproduce on current
main. They are follow-up base defects, not behaviors introduced by the current net diff, so I am no longer treating them as blockers unique to #442.
Validation on exact head d0de953: the focused MCP, identity, diagnostics, permissions, authority, and exclusion suite passed (235 tests); Ruff and bytecode compilation passed; all current GitHub checks are green. Agents Shipgate verification against b867f31 passed and its receipt reproduced as valid. Shipgate is static evidence only and does not supersede the code-review finding above.
…moval Addresses the P2 review finding on #442. The message this PR added offered the reader their one lever over an adapter they do not own: remove the `tool_sources` entry and the scan runs. That is a merge gate explaining how to stop gating a capability surface, and labelling it a workaround with a stated cost does not redeem it — the next scan simply omits those tools. Removed. Three further defects in the same message, all confirmed: - No typed `details["failure"]`. `input_parse_recovery()` routes on that key, so this fell through to the generic "inspect the file referenced in the error" — which here is a file inside someone else's package. It now carries `failure: "tool_source_mismatch"` with `source_type` and `configured_source_id`, and has a recovery branch. - It hardcoded `shipgate.yaml`, which is wrong for a nested or differently named manifest. - It pointed at `report.json`, which does not exist yet: this aborts during the scan. The route is worse than reported — `doctor` calls the same `_build_canonical_tools`, so it aborts here too and cannot be where the reader goes looking for the adapter. The sentence now names the adapter by `tool_sources[].type`, following the spelling `inputs/adapter_validation.py` already uses and the vocabulary sweep already pins. The route is a `review` action with no `path` — nothing in this repository is wrong, and a consumer routing on `path` would edit a file that was never the problem. The two repairs offered both keep the surface: upgrade Agents Shipgate or the package providing the adapter, or take an explicit human decision. Removal appears only as what it costs. Documented in `docs/errors.json` beside the other `details.failure` values. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Summary
codex_configtool source no longer aborts the scan on any MCP server it finds. Atool_sources[]row oftype: codex_configfailed outright withInputParseErrorwhenever the workspace contained a.mcp.jsonor.codex/config.tomldeclaring a server. Since a server with no enumerable tools still mints a<server>.*wildcard, any server at all reached it — the source type worked only on workspaces that had no MCP declarations to read, which is the opposite of what it is for.tool_sourcesentry produced the read, the message now names it, after the diagnosis and labelled as the workaround it is.The defect
inputs.mcp_manifestminted a per-server id for every tool (mcp_json:<server>,codex_config_mcp:<server>) and then wrapped a whole file's servers in oneLoadedToolSourcenamed for the file (codex_config_mcp:<path>).core.tool_identityrequires a loaded source to name the id its tools carry — a tool arriving under another source's name is a loader defect — so the two spellings could never agree.Reported for
.mcp.json;.codex/config.tomlwas equally broken (codex_config_mcp:<path>source holdingcodex_config_mcp:<server>tools). Both are fixed and both are parametrized in the regression test.Which side is wrong, and why
The loader's shape, not its ids. It now emits one
LoadedToolSourceper MCP server, built from the server, so there is no second place that spells the id.The other candidate repair — stamp the tools with the file-level id — is wrong.
_native_locatoris the bare tool name for MCP-like sources, so collapsing a file's servers into one source makes a legal.mcp.jsonwhose two servers both exposequeryfail with "defines the tool 'query' more than once": a false error with no repair. Verified directly, and pinned bytest_two_servers_in_one_file_may_expose_the_same_tool_name.The minted ids themselves are unchanged and stay deliberately path-free. I first also qualified them per file, to keep two packages that each declare
githubapart — that failstest_mcp_audit_same_adapter_pure_rename_is_not_a_capability_addition, because an MCP capability is its server and tool, so moving a.mcp.jsonmust not read as a capability addition. Reverted, and the rejected option is now pinned bytest_the_minted_server_id_stays_free_of_the_path_it_was_read_fromso it isn't rediscovered.Why no test caught it
The three existing
type: codex_configfixtures (test_verify_orchestrator.py:351,test_preflight.py:470,test_codex_boundary_check.py:757) all point at workspaces where the loader mints no tools, so nothing reached the check.Type
Verification
CI is authoritative for
python -m ruff check .,python -m compileall -q src tests, andpython -m pytest.Additional local checks run:
pytest -n auto -m "not perf" --ignore=tests/test_adapter_static_only.py, thentests/test_latency_budget.py -m perfandtests/test_adapter_static_only.pyseparately — all exit 0.ruff check src testsclean.src; the 8th is the rename-invariant pin that must pass both ways by design.mcp auditrename test../shipgate scan.Release-readiness notes
docs/checks.md— n/a, no check IDs changeSTABILITY.md— no version moves (report_schema_version,contract_version, every published schema document unchanged). Nothing that previously produced a report changes shape, because no workspace with an MCP server could produce one.detailsgainsconfigured_source_id(additive).Reviewer notes
Known gap, deliberately out of scope. Two files of the same kind declaring the same server name —
pkg_a/.mcp.jsonandpkg_b/.mcp.jsonboth declaringgithub, likely in a monorepo since server names are conventional — still fail:scanwith a duplicate-observation message naming a file the manifest never listed twice;mcp auditwith an unhandledConfigError: Duplicate capability ids are not allowedand no JSON envelope — this one is independent of this PR and reproduces onorigin/main.Both come from the same tension: path-free MCP capability identity and one-observation-per-identity are each deliberate. Reconciling them means deciding whether one repo declaring a server twice is one capability or two, and since the declarations may disagree, any merge rule is fail-open unless it records what it dropped. That is a design call for its own issue, not a drive-by inside this fix. Tracked separately.
No issue reference. No issue is filed for this bug. Happy to open one and add the citation to the comments if you'd like the usual provenance.
🤖 Generated with Claude Code