test: exercise DevSpace through Claude Code - #247
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR adds an explicit paid end-to-end acceptance test for the Claude Code → OAuth → MCP → DevSpace path and documents how to invoke it.
Confidence Score: 4/5The packaged host-test command needs to be removed from the package contract or made runnable from published contents before merging. The repository-local acceptance flow aligns with the server contracts, but the package check deliberately ships a command that immediately invokes unavailable source-tree build inputs and development dependencies. Files Needing Attention: scripts/test-package.mjs and package.json
|
| Filename | Overview |
|---|---|
| scripts/test-claude-host.mjs | Adds a bounded, isolated Claude-host OAuth/MCP acceptance harness with explicit process and temporary-state cleanup. |
| package.json | Adds the host-test command, but that command invokes development build tooling unavailable in a normal published-package installation. |
| scripts/test-package.mjs | Newly requires publishing the host-test source script despite the packaged artifact omitting the inputs needed by its command. |
| README.md | Documents the paid test's authentication prerequisite, isolation, tool restrictions, spend limit, cleanup, and CI exclusion. |
Sequence Diagram
sequenceDiagram
participant T as Host test
participant D as DevSpace server
participant O as OAuth endpoints
participant C as Claude Code
participant M as MCP endpoint
T->>D: Start isolated production server
T->>O: Register client
T->>O: Authorize with PKCE and owner token
O-->>T: Authorization code
T->>O: Exchange code for access token
O-->>T: Access token
T->>C: Run with restricted MCP configuration
C->>M: open_workspace
C->>M: read marker.txt
C-->>T: marker and workspaceId
T->>M: Verify read using workspaceId
Reviews (1): Last reviewed commit: "docs: describe the Claude host acceptanc..." | Re-trigger Greptile
| "dist/local-agent-daemon-main.js", | ||
| "dist/db/migrations.js", | ||
| "dist/ui/workspace-app.html", | ||
| "scripts/test-claude-host.mjs", |
There was a problem hiding this comment.
Published host test lacks build inputs
When a consumer runs the published test:host:claude script, it first invokes npm run build, but the package excludes the source and build configuration and production installations omit Vite and TypeScript, causing the command to fail before reaching the Claude acceptance flow.
This adds an opt-in acceptance test for the real Claude Code → OAuth → Streamable HTTP MCP → production DevSpace path. The harness gives Claude an unknown marker, exposes only
open_workspaceandread, and then independently reuses the returnedworkspaceIdthrough the MCP SDK. That makes the contract about an observable MCP outcome rather than trusting model prose.The run owns isolated roots, state, OAuth credentials, port, output limits, a 120-second deadline, and a $0.25 model budget. It terminates the exact processes it starts and removes its temporary files. The check stays outside the default suite and CI because it requires an authenticated, paid Claude Code installation. The package acceptance test also proves the published package contains the host script.
Verified locally with
npm test(109 passed, one macOS-only skip),npm run typecheck,npm run test:package, syntax/diff checks, and the missing-executable cleanup path. A live host attempt completed the build, production server startup, and OAuth exchange, then stopped before inference because the local Claude Code installation is logged out; reported model cost was $0.This PR is stacked on #239.