[songoow]

Add an independent native Claude Code entrypoint for file-submission research trials. It installs an explicitly hashed/versioned native binary, reuses SForge's Claude Stop hook and session recovery, and keeps submission, grading and the execution deadline with the native provider. Explicit permissions make the hook and submission script usable under umask 077; the Docker transport assigns the uploaded prompt to the agent with mode 0600 even when host and container UIDs differ.

The runner requires a clean declared source revision and an Anthropic-compatible endpoint, pins auxiliary model aliases to the selected model, retains structured execution evidence, and returns failure for incomplete/failed execution. Existing runner paths are unchanged. The entrypoint introduces no LoopX Goal/Todo controller or automatic cohort launcher. The guide explains host/container judge URL routing.

Validation at 81d84aa:

• Four focused tests passed, including a real disposable Docker upload with mismatched host/container UIDs. The agent can read its prompt; unrelated private uploads and access by another container user remain restricted.
• Ruff and diff checks passed. The earlier revision also passed compilation, CLI help, real native streamed tools, exact-session resume, Stop continuation, native Goal evaluation and Plan-to-execution checks.
• An earlier synthetic native Docker submission returned its expected score and respected the provider deadline. An initial synthetic judge working-directory failure and later host/container URL and prompt-ownership failures remain recorded as failed qualifications; the permission regression now has a focused real-container test.
• Raw logs, credentials and local experiment state are excluded.

Scope: research transport only; native CLI effort values remain a local provider vocabulary. The semantic advisory does not analyze benchmark sources. Reuse of the existing provider avoids a parallel continuation implementation. Short checks do not establish long-run reliability, gateway usage-accounting accuracy or benchmark improvement. No merge or long-run qualification is claimed.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-22T16:29:58.916554Z	2b306c3	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2b306c3042

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Skip native-provider tests when SForge is unavailable

In the standard test environment, these unconditional imports abort pytest collection because SForge is not included in either the project dependencies or the test extra. The test shards in .github/workflows/python-tests.yml install .[test] and collect the whole repository, so this also breaks the existing CI suite without opting into the new runner. Running python -m pytest benchmark/tests/test_native_claude.py -q reproduces ModuleNotFoundError: No module named 'sforge'. Guard the optional provider imports with pytest.importorskip, as the existing SForge tests do, before importing NativeClaudeWorker.

Useful? React with 👍 / 👎.

[loopx-agent]

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

REQUEST_CHANGES at 81d84aaa67385730992c1262cabef93934883236。 有一项已独立复现的 P1：未选择此 runner、未安装可选 SForge 的常规测试环境无法完成收集。已读取现有机器人提出的同一问题；以下是完整精确头审查及独立验证，不另发重复 inline。

动机

需要用独立原生 Claude Code 执行文件提交研究试验的操作者。目前既有 runner 没有这个明确固定二进制版本的入口；本提案复用原生会话、Stop hook 和 SForge 计分，减少为新宿主另建执行循环的成本。预期用户能显式选定模型、版本和预算，运行一次隔离试验并读取真实执行与评分记录。该入口仍是研究 transport，不能据此声称长期可靠性或 benchmark 收益，也不应影响未选择它的开发者。

已有入口的缺口值得补齐；独立 provider 适配比再建 LoopX Goal 控制器更合适。不过当前头把可选运行环境变成所有测试收集者的隐性必需条件，实际扩大了影响范围。这个回归需要先修复，不能因为四项配置了依赖的测试通过而忽略。

改动思路

现有 source bootstrap 冻结 SForge，现有 clean-revision 检查绑定研究源码；NativeClaudeWorker 只负责本地二进制校验、上传、权限与模型参数。原生 Claude/SForge 继续拥有会话恢复、Stop 协议、工具、提交、评分和执行期限，没有第二套编排状态。固定prompt之外的上传沿用原模式；没有增加通用chmod或扩展数据读取范围。Python 放在已有 provider transport 有明确理由，不需要为了本次适配改写通用控制平面。

已重新检查17:02的新提交：普通CLI改用一个仅处理固定prompt目标的NativeClaudeDockerBackend，上传后归agent所有并保持0600。相关源码/调用和跨UID测试已重验；旧默认依赖阻断仍存在。

当前 PR 只提供独立研究 transport、原生二进制/参数适配和对应测试文档；既有 runner 与实验保持原 owner。

本轮最强反例是“功能显式选择，却在关闭路径破坏普通测试”。同时检查了实际 Docker 安装与普通 CLI 的失败记录。下表描述当前源码和本轮观察，不能当作真实模型完成了任务：

输入/边界	实际观察
未安装 SForge，收集 benchmark/tests	基线退出0；当前头退出2，新增模块导入失败
显式冻结 SForge，隔离 Docker，umask077	四项测试通过，agent 可以执行 hook 和提交入口
普通 CLI、合成可执行文件、真实 Docker/provider	已实际到达执行并保留 unknown/harness_error，CLI退出1；完整成功评分链未取得

具体改动

规格依据为 docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.md，不可变版本 e6a862958e219c111c0830074044b48c47e2a7aa。按 C0: Reproduction and adapter fidelity，固定原生 runner、生命周期与结果边界是资格要求；当前二进制/权限 seam 有实证，完整原生运行资格仍有缺口。按 6.5 Benchmark integrity and anti-cheating，计分、隐藏材料和权限归原生 owner；本轮仅使用合成任务、合成二进制和独立 disposable 环境，无模型调用或真实研究试验。另读同版本 benchmark/README.md 的可复用 provider/原生权威边界，不能将实验接入改成默认依赖。

关键代码讲解

• NativeClaudeWorker.__init__/_verify_binary 要求明确模型、局部 effort、endpoint/credential、digest/version，再校验实际文件；install 前再次检查，避免文件替换后仍沿用旧声明。漂移反例在复制前拒绝。
• augment_env/format_run_cmd 将辅助模型别名绑定同一选择，并在初次与 continue 命令都追加 effort。这里复用 provider 的命令和联网工具限制；配置相等仍不证明网关按该模型实际计费。
• install_stop_hook 在容器内以 agent 验证 digest/version，再复用原 Stop hook；显式0755修复私有umask留下的脚本读取权限，settings保持0600。实际上传测试通过，未把宿主或个人 Claude 设置改成持续执行。
• main 经 source lock、runner pin、文件任务校验进入原生 run_agent，保存 claude-result，要求实际 runtime、complete grading 和可接受 execution status 才返回成功。没有仅以评分存在判完成。

对主干的风险

[P1] 可选 provider 的无条件 import 破坏默认测试收集。 新增测试12–13行 在 skip 标记执行之前导入 sforge 和适配器。项目 test extra 不包含 SForge；本地在新头与基线使用相同 uv run --extra test python -m pytest --collect-only -q benchmark/tests 在不可变基线收集1101项并退出0，当前头收集同一批后因 ModuleNotFoundError: No module named 'sforge' 退出2。仓库 workflow 源码明确对整个 collection 分片，所以未启用新 runner 也受影响。这里没有查询、等待或推断远端 CI 状态。

最小修复是采用相邻可选 provider 测试的 pytest.importorskip("sforge")，放在两次 provider 导入之前；保留显式提供依赖时的四项测试。重验上述常规收集和显式 SForge/Docker 测试，不能通过全局安装依赖、排除整目录或删除有效回归来掩盖隔离失败。已有 同一 inline 保留，不需要重复留言。

当前新头的四项显式 provider 测试重新通过，包含不同宿主/容器UID下prompt可读、unrelated-private仍拒绝、nobody拒绝；原生参数、hook与digest测试同时保留。真实 CLI help 的原检查对应未变的参数定义。标准本地 premerge 的4项直接检查、compile、10项catalog、8项risk及公私边界均通过，但保留原生 benchmark_sensitive 人工 hold，整体不称可合并。旧头2b306c的更完整本地 CLI/Docker 探针在 provider e8d10a887d5173382d484afd4b3f2f2e86e8a206 的 Docker SDK stream.close 遇到 ENOTCONN，保存 unknown/harness_error 并返回1；这是未改动的外部 provider 路径，本轮没有伪装修复或归罪给新适配器。首次 Docker build 的缺失本地内容摘要也保留，改用独立容器准备环境后才取得这条失败观察。成功评分、resume与deadline全链本轮独立资格仍未知，作者的模型/Goal/Plan声明没有被冒充为我的执行。新头没有修复该外部stream owner，本次不继承旧头运行或批准。

我的整体评价

当前请求修改。long_horizon 为 not_yet_proven：适配边界与失败保存合理，但成功的完整运行及续作未独立取得；user_experience 为 regression：未选择功能的开发者仍被阻断测试收集。局部 effort 词汇留在 provider 内，未新增共享状态或通用权限协议；主体机制与问题相称，最有价值的有界修正是恢复可选依赖隔离。需要依照修复后的完整新头重新判断，原生评分和长期收益分开证明。

已检查研究 CLI、文档和测试入口；此 PR 没有渲染页面变更，视觉评审不适用。本轮未独立验证真实 Claude 模型、完整评分成功、长期可靠性或安装后的产品界面。此次审查没有合并、安装共享 CLI、运行付费模型或启动研究 cohort。

English verdict: REQUEST_CHANGES at 81d84aaa67385730992c1262cabef93934883236. Default collection passes on the base but fails on the head because optional SForge imports are unconditional. Four explicit-provider/Docker tests pass. The full synthetic CLI path retains a truthful unknown failure from unchanged provider streaming; successful end-to-end qualification is not claimed. Restore optional-dependency isolation and rerun both environments.