Skip to content

test: exercise DevSpace through Claude Code - #247

Closed
Waishnav wants to merge 2 commits into
codex/test-suite-overhaulfrom
codex/claude-mcp-host-e2e
Closed

test: exercise DevSpace through Claude Code#247
Waishnav wants to merge 2 commits into
codex/test-suite-overhaulfrom
codex/claude-mcp-host-e2e

Conversation

@Waishnav

@Waishnav Waishnav commented Aug 23, 2026

Copy link
Copy Markdown
Owner

This adds an opt-in acceptance test for the real Claude Code → OAuth → Streamable HTTP MCP → production DevSpace path. The harness gives Claude an unknown marker, exposes only open_workspace and read, and then independently reuses the returned workspaceId through the MCP SDK. That makes the contract about an observable MCP outcome rather than trusting model prose.

The run owns isolated roots, state, OAuth credentials, port, output limits, a 120-second deadline, and a $0.25 model budget. It terminates the exact processes it starts and removes its temporary files. The check stays outside the default suite and CI because it requires an authenticated, paid Claude Code installation. The package acceptance test also proves the published package contains the host script.

Verified locally with npm test (109 passed, one macOS-only skip), npm run typecheck, npm run test:package, syntax/diff checks, and the missing-executable cleanup path. A live host attempt completed the build, production server startup, and OAuth exchange, then stopped before inference because the local Claude Code installation is logged out; reported model cost was $0.

This PR is stacked on #239.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a364a794-cfb0-4ecc-a421-41415d09de2b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Waishnav Waishnav changed the title codex/claude mcp host e2e test: exercise DevSpace through Claude Code Aug 23, 2026
@Waishnav
Waishnav marked this pull request as draft August 23, 2026 20:46
@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds an explicit paid end-to-end acceptance test for the Claude Code → OAuth → MCP → DevSpace path and documents how to invoke it.

  • Starts an isolated production server and performs a real OAuth authorization-code exchange.
  • Restricts Claude to open_workspace and read, verifies an unknown marker through MCP, and bounds spend, output, and runtime.
  • Adds the harness to the npm package acceptance list, although the resulting published command lacks its build inputs.

Confidence Score: 4/5

The packaged host-test command needs to be removed from the package contract or made runnable from published contents before merging.

The repository-local acceptance flow aligns with the server contracts, but the package check deliberately ships a command that immediately invokes unavailable source-tree build inputs and development dependencies.

Files Needing Attention: scripts/test-package.mjs and package.json

Important Files Changed

Filename Overview
scripts/test-claude-host.mjs Adds a bounded, isolated Claude-host OAuth/MCP acceptance harness with explicit process and temporary-state cleanup.
package.json Adds the host-test command, but that command invokes development build tooling unavailable in a normal published-package installation.
scripts/test-package.mjs Newly requires publishing the host-test source script despite the packaged artifact omitting the inputs needed by its command.
README.md Documents the paid test's authentication prerequisite, isolation, tool restrictions, spend limit, cleanup, and CI exclusion.

Sequence Diagram

sequenceDiagram
  participant T as Host test
  participant D as DevSpace server
  participant O as OAuth endpoints
  participant C as Claude Code
  participant M as MCP endpoint
  T->>D: Start isolated production server
  T->>O: Register client
  T->>O: Authorize with PKCE and owner token
  O-->>T: Authorization code
  T->>O: Exchange code for access token
  O-->>T: Access token
  T->>C: Run with restricted MCP configuration
  C->>M: open_workspace
  C->>M: read marker.txt
  C-->>T: marker and workspaceId
  T->>M: Verify read using workspaceId
Loading

Reviews (1): Last reviewed commit: "docs: describe the Claude host acceptanc..." | Re-trigger Greptile

Comment thread scripts/test-package.mjs
"dist/local-agent-daemon-main.js",
"dist/db/migrations.js",
"dist/ui/workspace-app.html",
"scripts/test-claude-host.mjs",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Published host test lacks build inputs

When a consumer runs the published test:host:claude script, it first invokes npm run build, but the package excludes the source and build configuration and production installations omit Vite and TypeScript, causing the command to fail before reaching the Claude acceptance flow.

@Waishnav Waishnav closed this Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant