Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 49 additions & 8 deletions ops/DRIVE-LOCAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ submits commands through the existing local launcher. The daemon journals those
pins before implementation starts. Running the YAML template directly refuses
the snapshot step because its pinned inputs are missing.

`gate-snapshot` runs first. It extracts the package helper, verifier and SDK
`gate-snapshot` runs first. It extracts the package helper, verifier, acceptance helper and SDK
picker source with `git --no-replace-objects show <head>:<path>`, then compiles
that picker in a temporary directory outside the checkout. Its own extraction
script also comes from that Git ref. Neither working-tree helper files nor
Expand All @@ -38,7 +38,44 @@ executed at the repository root with a two-minute bound. For example:

Multiple `Verify:` lines mean all commands must pass. Commands come from the
backlog before implementation; the agent cannot substitute its own acceptance
checks in package.json. Entries with no executable checks are skipped with
checks in package.json. Existing inline Node assertions (`node -e`, with an
optional `--input-type=module`) remain supported: their code is part of the
pinned backlog. Script checks must declare their code inputs explicitly:

```text
- **Fix the value** Update `src/value.txt` to contain exactly "fixed".
Verify: {"argv":["node","checks/value.cjs"],"inputs":[{"path":"checks/value.cjs","ref":"HEAD"}]}
```

`HEAD` is resolved once to the launcher's full commit ID. An explicit full
commit ID is also accepted; branch names and missing Git objects are refused.
Git inputs must name regular files at repository-relative paths. Verification
extracts their Git bytes into a fresh temporary directory outside the checkout,
preserving relative paths, and replaces matching argv elements with extracted
paths. Declare assertion dependencies in the same `inputs` array so they are
extracted together. The command still runs with the implementation checkout as
its working directory, so assertions can read changed source and artifacts.

Alternatively, use an absolute path without `ref` for an externally owned
script, for example `{"argv":["node","/opt/acceptance/value.cjs"],"inputs":[{"path":"/opt/acceptance/value.cjs"}]}`.
Such files must exist outside the implementation's declared write scope.
Validation checks both real paths and symlink aliases beneath writable
directories; hard-linked external inputs are refused because their ownership
cannot be established from a path. A relative path without a pin, an undeclared script, an unavailable
pin, or an external file inside write scope fails with
`ACCEPTANCE_IMMUTABILITY_VIOLATION` before any check executes. Node script checks
use the launcher's Node binary; other entry points must be declared scripts
with a shebang (for example `checks/value.sh`). Declaring `/bin/sh` does not
authorize an arbitrary `-c` command. Ambient `NODE_OPTIONS` and `NODE_PATH`
cannot preload checkout code.

The selected package retains `verificationCommands` and adds
`verification: {ref, checks: [{argv, inputs}]}`. Every scope, verification and
report operation reconstructs both fields from the original pinned backlog.
The initial scope step therefore refuses an invalid contract before handing
the package to implementation.

Entries with no executable checks are skipped with
`missing_executable_checks`, alongside the existing unbounded/stale scope
reasons. The old F8b entry now carries a source assertion for its declared
rename. That assertion also allows an already-completed package to pass
Expand All @@ -60,7 +97,8 @@ Verification reconstructs the selected work from the unchanged backlog and
compares its scope and commands to package.json. Each package check must pass
before the SDK regression suite runs. An unchanged implementation whose DoD
is unmet fails. HEAD/branch changes fail, and out-of-scope changes produced by
an acceptance command fail too. A failed scope or verification step prevents
an acceptance command fail too. `PACKAGE_VERIFIED` is emitted only after that
post-check validation succeeds. Refusals include the package's DoD. A failed scope or verification step prevents
the dependent report step from running.

The scope command runs again after the SDK build and suite, before reporting.
Expand All @@ -74,14 +112,17 @@ The execution contract has one owner for each kind of data:
| HEAD, branch, backlog hash | Preparing launcher pins the original commit/branch and the hash of its committed backlog in submitted commands. The package records the commit as `head`; no adjacent ref or checksum file is authority. |
| Package metadata | Private atomic JSON artifact for the agent; scope, verify and report reconstruct its fields from the picker built from the pinned ref and the pinned backlog. It cannot redefine the original HEAD. |
| Allowed paths and protected paths | `local-work-verification.mjs` extracted from the pinned ref; index and working tree checked separately against the original HEAD, including non-ignored untracked files. |
| Acceptance argv | Parsed from the pinned backlog; every command must succeed, with scope rechecked after execution. |
| Acceptance scripts and dependencies | **Open blocker:** arbitrary argv can load mutable source from implementation scope. |
| Acceptance argv | Parsed from the pinned backlog and reconstructed as the package's `verification` contract; every command must succeed, with scope rechecked before emitting `PACKAGE_VERIFIED`. |
| Acceptance scripts and declared dependencies | Explicit `inputs` load regular files from full Git commit IDs or absolute external paths validated outside implementation write scope. Undeclared script entry points are refused. |

**A same-user agent can still write to the temporary execution directory.**
This meets the narrower bar that gate inputs come from a pinned Git ref rather
than files implementation edits. It does not make extracted runtime files
immutable, isolate processes, or prevent arbitrary Git-storage tampering.

The remaining acceptance-input decision is documented in
`runtime-evidence/drive-threads-0909-decisions.md`. This flow is not ready for
unattended use until those inputs have an explicit enforced ownership contract.
Acceptance authors still own the assertion program and its dependency
declarations. This is an input ownership contract, not analysis or isolation of
arbitrary programs: trusted checks must not delegate assertions to undeclared
mutable code via inline evaluation, subprocesses, or dynamically computed paths.
The historical acceptance-input decision is documented in
`runtime-evidence/drive-threads-0909-decisions.md`.
24 changes: 19 additions & 5 deletions ops/drive-local-flow.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,21 @@ import { spawnSync } from 'node:child_process';
import { chmodSync, existsSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { join, resolve } from 'node:path';
import test from 'node:test';
import { fixture, packagePath } from './local-work-test-fixture.mjs';
import { socketPathFor } from '../packages/sdk/dist/daemon-connection.js';
import { entry, fixture, packagePath } from './local-work-test-fixture.mjs';

const quote = text => "'" + text.replaceAll("'", "'\\''") + "'";
for (const scenario of ['outside edit', 'verifier edit', 'unchanged package', 'HEAD repin', 'forged snapshot']) {
for (const scenario of ['outside edit', 'verifier edit', 'unchanged package', 'HEAD repin', 'forged snapshot',
'edited pinned acceptance', 'undeclared acceptance']) {
test(`drive-local journals failure and blocks reporting for ${scenario}`, t => {
const f = fixture(t);
const acceptance = scenario.includes('acceptance');
const argv = ['node', 'src/check.cjs'];
const check = scenario === 'undeclared acceptance' ? argv : {
argv, inputs: [{ path: 'src/check.cjs', ref: 'HEAD' }],
};
const f = acceptance ? fixture(t, entry('Fix value', 'src/', [check]), {
'src/check.cjs': "require('node:assert/strict').equal(require('node:fs').readFileSync('src/value.txt','utf8'),'fixed');",
}) : fixture(t);
const wrapper = resolve('testdata/preflight/wrapper-session.mjs');
const cli = join(f.root, '.relayflow/agent.mjs');
f.put('.relayflow/agent.mjs', `#!/usr/bin/env node
Expand All @@ -19,6 +28,7 @@ const request = await receiveWrapperRequest();
if (request) {
${scenario === 'outside edit' ? `writeFileSync(${JSON.stringify(join(f.root, 'outside.txt'))}, 'changed');` : ''}
${scenario === 'verifier edit' ? `writeFileSync(${JSON.stringify(join(f.root, 'ops/local-work-package.mjs'))}, 'process.exit(0)');` : ''}
${scenario === 'edited pinned acceptance' ? `writeFileSync(${JSON.stringify(join(f.root, 'src/check.cjs'))}, 'process.exit(0)');` : ''}
${scenario === 'HEAD repin' ? `
const git = (...args) => execFileSync('git', args, {cwd: ${JSON.stringify(f.root)}, encoding: 'utf8'}).trim();
writeFileSync(${JSON.stringify(join(f.root, 'outside.txt'))}, 'changed');
Expand Down Expand Up @@ -56,12 +66,16 @@ if (request) {
assert(dataDir, result.stderr);
t.after(() => rmSync(dataDir, { recursive: true, force: true }));
const journal = readFileSync(join(dataDir, 'journal.jsonl'), 'utf8').trim().split('\n').map(JSON.parse);
const failedStep = ['unchanged package', 'forged snapshot'].includes(scenario) ? 'verify' : 'scope';
const failedStep = scenario === 'undeclared acceptance' ? 'initial-scope' :
['unchanged package', 'forged snapshot', 'edited pinned acceptance'].includes(scenario) ? 'verify' : 'scope';
const completion = journal.find(e => e.entry_type === 'step.completed' && e.step_id === failedStep);
assert.equal(completion?.payload.verification.verdict, 'fail', JSON.stringify(journal));
assert.equal(completion.payload.completionReason, 'retries_exhausted');
assert.equal(completion.payload.verification.detail, 'exit code was 1');
assert(!journal.some(e => e.entry_type === 'step.attempt.started' && e.step_id === 'report'));
assert(!existsSync(join(dataDir, 'relayflowd.sock')));
if (scenario === 'undeclared acceptance') {
assert(!journal.some(e => e.entry_type === 'step.attempt.started' && e.step_id === 'implement'));
}
assert(!existsSync(socketPathFor(dataDir)));
});
}
154 changes: 154 additions & 0 deletions ops/local-work-acceptance.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
// Acceptance code has a separate ownership contract from implementation data.
// Git inputs are extracted afresh; external inputs must be outside write scope.
import assert from 'node:assert/strict';
import { execFileSync, spawnSync } from 'node:child_process';
import { chmodSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
import { dirname, isAbsolute, join, resolve, sep } from 'node:path';
import { tmpdir } from 'node:os';

const violation = detail => `ACCEPTANCE_IMMUTABILITY_VIOLATION: ${detail}`;
const within = (path, root) => path === root || path.startsWith(`${root}${sep}`);
const git = (...args) => execFileSync('git', ['--no-replace-objects', ...args], { stdio: ['ignore', 'pipe', 'pipe'] });

export function acceptanceContract(body, ref) {
const checks = body.split('\n').filter(line => /^\s*Verify:/.test(line)).map(line => {
let declaration;
try { declaration = JSON.parse(line.replace(/^\s*Verify:\s*/, '')); }
catch { throw new Error('INVALID_EXECUTABLE_CHECK: Verify must contain JSON argv or {argv, inputs}'); }
assert(Array.isArray(declaration) || (declaration && typeof declaration === 'object' &&
Object.keys(declaration).every(key => key === 'argv' || key === 'inputs')),
'INVALID_EXECUTABLE_CHECK: expected argv or {argv, inputs}');
const { argv, inputs = [] } = Array.isArray(declaration) ? { argv: declaration } : declaration ?? {};
assert(Array.isArray(argv) && argv.length > 0 &&
argv.every(arg => typeof arg === 'string' && !arg.includes('\0')) && argv[0].trim(),
'INVALID_EXECUTABLE_CHECK: expected nonempty command argv');
assert(Array.isArray(inputs), violation('inputs must be an array'));
return { argv, inputs: inputs.map(input => {
assert(input && typeof input.path === 'string' && input.path && !input.path.includes('\0'),
violation('each input needs a path'));
assert(Object.keys(input).every(key => key === 'path' || key === 'ref'), violation('unknown input field'));
return { ...input, ...(input.ref === 'HEAD' ? { ref } : {}) };
}) };
});
return { ref, checks };
}

// Include symlink aliases beneath a directory scope: src/link -> checks grants
// implementation a write route to checks even if checks is lexically outside src.
function writablePaths(scopes) {
const paths = [];
const visited = new Set();
function visit(path) {
paths.push(resolve(path));
let target;
try { target = realpathSync(path); }
catch (error) {
if (error.code === 'ENOENT' || error.code === 'ENOTDIR') return;
throw error;
}
paths.push(target);
if (visited.has(target)) return;
visited.add(target);
if (lstatSync(target).isDirectory()) {
for (const entry of readdirSync(target)) visit(join(target, entry));
}
}
for (const scope of scopes) visit(scope);
return paths;
}

function validateInput(input, writable) {
if (input.ref !== undefined) {
assert(/^[a-f0-9]{40}$/.test(input.ref), violation(`missing pinned Git ref for ${input.path}`));
assert(!isAbsolute(input.path) && input.path.split('/').every(part => part && part !== '.' && part !== '..'),
violation(`Git input must be a repository-relative path: ${input.path}`));
try {
assert.equal(git('cat-file', '-t', input.ref).toString().trim(), 'commit');
// Do not materialize a symlink blob as executable source.
const mode = git('ls-tree', input.ref, '--', input.path).toString().split(' ')[0];
assert(mode === '100644' || mode === '100755');
return git('show', `${input.ref}:${input.path}`);
} catch (cause) {
throw new Error(violation(`pinned Git input unavailable: ${input.ref}:${input.path}`), { cause });
}
}
assert(isAbsolute(input.path), violation(`missing pinned Git ref for ${input.path}; external inputs require absolute paths`));
let target;
try { target = realpathSync(input.path); }
catch (cause) { throw new Error(violation(`external input unavailable: ${input.path}`), { cause }); }
const stat = lstatSync(target);
assert(stat.isFile(), violation(`external input must be a file: ${input.path}`));
assert(stat.nlink === 1, violation(`external input has writable hard-link aliases: ${input.path}`));
assert(!writable.some(scope => within(resolve(input.path), scope) || within(target, scope)),
violation(`acceptance input is implementation-writable: ${input.path}`));
return null;
}

export function validateAcceptance(pkg) {
const contract = pkg.verification;
assert(contract && /^[a-f0-9]{40}$/.test(contract.ref) && contract.ref === pkg.head,
violation('missing or changed verification pinned Git ref'));
assert(contract.checks?.length > 0, 'MISSING_EXECUTABLE_CHECKS');
assert.deepEqual(contract.checks.map(check => check.argv), pkg.verificationCommands,
violation('argv does not match the verification contract'));
const writable = writablePaths(pkg.filesInScope);
return contract.checks.map(check => {
const paths = new Set();
const sources = check.inputs.map(input => {
assert(!paths.has(input.path), violation(`duplicate acceptance input: ${input.path}`));
paths.add(input.path);
return validateInput(input, writable);
});
const argv = check.argv;
// Legacy inline Node assertions are themselves code pinned in the backlog.
// Runtime options that could preload checkout code are deliberately excluded.
const node = argv[0] === 'node' || argv[0] === process.execPath;
const inline = node && ((argv[1] === '-e' && argv.length === 3) ||
(argv[1] === '--input-type=module' && argv[2] === '-e' && argv.length === 4));
const script = node ? argv[1] : argv[0];
if (!inline) {
assert(script && !script.startsWith('-'), violation(`unsupported acceptance invocation: ${JSON.stringify(argv)}`));
const input = check.inputs.find(input => input.path === script);
if (!input) {
assert(!writable.some(scope => within(resolve(script), scope)),
violation(`acceptance script is implementation-writable: ${script}`));
throw new Error(violation(`undeclared acceptance script: ${script}; declare a pinned Git input or external absolute path`));
}
if (!node) {
const source = sources[check.inputs.indexOf(input)] ?? readFileSync(input.path);
assert(source.subarray(0, 2).toString() === '#!',
violation(`declare an acceptance script with a shebang, not a runtime executable: ${script}`));
}
}
return { ...check, sources, node };
});
}

export function runAcceptance(pkg) {
// Validate every input before executing any command, including later checks.
const checks = validateAcceptance(pkg);
for (const check of checks) {
const directory = mkdtempSync(join(tmpdir(), 'drive-acceptance-'));
try {
const replacements = new Map();
check.inputs.forEach((input, index) => {
if (check.sources[index] === null) return;
const path = join(directory, input.path);
mkdirSync(dirname(path), { recursive: true });
writeFileSync(path, check.sources[index], { flag: 'wx', mode: 0o600 });
if (check.argv[0] === input.path) chmodSync(path, 0o700);
replacements.set(input.path, path);
});
const argv = check.argv.map(arg => replacements.get(arg) ?? arg);
if (check.node) argv[0] = process.execPath;
console.log(`CHECK ${JSON.stringify(check.argv)}`);
// Do not let ambient Node preload or search-path settings add mutable code.
const env = { ...process.env };
delete env.NODE_OPTIONS;
delete env.NODE_PATH;
const result = spawnSync(argv[0], argv.slice(1), { stdio: 'inherit', timeout: 120000, env });
assert(!result.error && result.status === 0,
`PACKAGE_CHECK_FAILED: ${JSON.stringify(check.argv)} (${result.error?.message ?? result.signal ?? result.status}); DoD: ${pkg.definitionOfDone.join('; ')}`);
} finally { rmSync(directory, { recursive: true, force: true }); }
}
}
Loading
Loading