External Black-Box Evaluation: ClawRouter x402 Payment Flow #436

Hi BlockRunAI team,

We recently ran a limited external black-box evaluation of ClawRouter using an adversarial payment/execution test methodology.

The purpose was not to inspect or modify ClawRouter internals, but to observe what an external participant can establish from the protocol boundary when payment, execution and transport outcomes diverge.

What we observed

1. Payment accepted, followed by transport-level failure

In one adversarial scenario, the test gateway accepted the payment and the subsequent client interaction eventually ended with a generic fetch failure / premature connection close.

The important observation is that the client-visible failure did not expose the economic state of the request.

In other words, from the external client's perspective:

payment accepted → execution outcome not established → transport failure

The client-visible error alone is therefore insufficient to determine whether the paid execution happened, failed, or remained unresolved.

2. Repeated paid requests produced independent executions

We also exercised repeated paid requests against the gateway.

The observed requests resulted in independent economic executions. The probe did not attach an idempotency key to these repeated requests, so this should not be interpreted as proof of an idempotency violation for a single logical operation.

Rather, the observation is:

repeated paid requests can produce separate executions when no idempotency identity is supplied.

This is relevant to retry/recovery design because a client may not be able to safely distinguish “retry the operation” from “start another paid operation” without an explicit operation identity.

Why this is interesting

This evaluation was specifically looking at the boundary between:

• payment acceptance
• request execution
• transport response
• proof that the requested work actually completed

A successful payment and a successful HTTP exchange are not necessarily equivalent to proof of completed work.

Scope and limitations

• Black-box testing only.
• No access to ClawRouter internals or private state.
• The test used a controlled gateway for deterministic failure injection.
• We are reporting externally observable client behavior, not claiming an internal ClawRouter defect.
• The timeout probe was intentionally adversarial and should not be interpreted as normal production traffic.
• The repeated-request observation used separate requests without an idempotency key.

Raw evidence and methodology are available here:

• Argus Agent Test Lab: https://github.com/igor-vii/Argus-Agent-Test-Lab
• Evidence package: PR Scoring receives empty prompt — OpenClaw sends content as array, not string #60 (a1-external-subjects-evidence)
• Test methodology: ARGUS_EXTERNAL_BLACKBOX_TEST_PROTOCOL.md

We'd be interested in whether these observations match the guarantees you intend ClawRouter to provide around paid execution, retries and unresolved outcomes.

If useful, we'd be happy to run a second pass against a specific expected behavior or contract.