drive: cloud run a4980bfe

ops/NEXT.md

@@ -1,98 +1,49 @@
-# NEXT — WP-GATE3-BACKLOG-PICKER: First honest step toward Software Garden
+# NEXT — Gate 3: Build the work package consumer
 
-**Target gate:** Gate 3 (per ops/TARGET.md — this run is pinned to gate 3 only)
-
-**Work package:** WP-GATE3-BACKLOG-PICKER — Build a flow that reads ops/BACKLOG.md and emits a structured work package
+**Scope:** Gate 3 — close the Garden's loop. CODE task, SDK-side. The picker EMITS a work package (sdk/src/backlog-picker.ts + testdata/backlog-picker.flow.yaml, merged, four tested properties) and NOTHING consumes it — that is the missing half. Build the consumer: an SDK entrypoint taking an emitted package and turning it into something runnable, validating it has a title, a non-empty scope and a definition of done, and REFUSING with a typed reason when it does not, because a package that cannot be verified must not become work. NOTE: two previous attempts (ee5c9b3e, 06c0d6ab) did this correctly and their files were LOST before delivery by a platform fault — the build sandbox's .git points at a directory that does not exist, so writes cannot be captured. You are not duplicating live work.
 
 ## Objective
 
-Gate 3 per RFC-0001 §3 is "a relayflow can power a factory → Software Garden." The done-when is a labeled issue flowing to a reviewed PR end-to-end. ops/TARGET.md directs: "Do the SMALLEST honest first step, not the whole thing. Good candidate: a flow that reads ops/BACKLOG.md, picks one entry by a deterministic rule, and emits a structured work package."
-
-**This is that smallest step**: a flow file that demonstrates gate-3 machinery (flows that build and improve other flows) without attempting the full discover→implement→review→merge DAG.
-
-## Current state
+Build the SDK entrypoint that takes an emitted work package and turns it into something runnable. The consumer must validate that the package has:
+- A title (non-empty string)
+- A non-empty scope
+- A definition of done
 
-- Gates 1 (GREEN) and 2 (AMBER, in progress) have working primitives on main
-- Gate 3 is RED (not started) per ops/SCOREBOARD.md
-- ops/BACKLOG.md exists with structured entries
-- No gate-3 flows exist yet
+When any of these is missing or invalid, the consumer REFUSES with a typed reason. A package that cannot be verified must not become work.
 
 ## Files in scope
 
-**New files to create:**
-- `testdata/backlog-picker.flow.yaml` — the flow spec that reads ops/BACKLOG.md and selects one entry
-- `testdata/backlog-picker.spec.canonical.json` — canonical compiled spec (via `flows check`)
-- `sdk/tests/backlog-picker.test.ts` — test proving deterministic selection given the same input
-
-**Files to modify:**
-- None required for the minimal step
-
-## Definition of done (all three required per ops/TARGET.md)
-
-1. **`flows check` resolves the flow** — `testdata/backlog-picker.flow.yaml` passes preflight validation with exit 0
-
-2. **A test proves selection is deterministic** — `sdk/tests/backlog-picker.test.ts` demonstrates that:
-   - Given the same ops/BACKLOG.md content, the flow always selects the same entry
-   - The selection rule is deterministic and documented (e.g., "first non-done entry", "alphabetically first", or similar)
-   - The test verifies structured output (the work package emitted has required fields)
-
-3. **`cd sdk && npm test` green** — all SDK tests pass including the new backlog-picker test
-
-## Implementation approach
-
-The SMALLEST working implementation:
-
-- **Flow structure:**
-  ```yaml
-  spec_version: "1.0"
-  name: backlog-picker
-  steps:
-    - name: read-backlog
-      type: deterministic
-      command: "cat ops/BACKLOG.md"
-    - name: select-entry
-      type: deterministic
-      command: # deterministic selection logic (e.g., first non-done entry)
-    - name: emit-package
-      type: deterministic
-      command: # output structured work package JSON
-  ```
-
-- **Deterministic rule examples:**
-  - First entry in the file
-  - First entry matching a pattern
-  - Alphabetically sorted first
-  - Line-number based
-
-  Choose the simplest that is defensible as "deterministic"
-
-- **Output format:** Structured JSON work package with fields like:
-  - `title`: work package name
-  - `description`: what needs to be done
-  - `files_in_scope`: estimated file paths
-  - `gate`: which gate this serves
-
-## Explicitly OUT of scope
-
-- LLM or agent steps (gate 1 is deterministic-only for now)
-- GitHub integration (creating actual issues or PRs)
-- The full discover→implement→review→merge DAG
-- Issue labeling, assignment, or tracking
-- Integration with the existing drive.yaml workflow
-- Any changes to kernel code
-- Changes to gates 1, 2, or 4-9
-- ops/FORBIDDEN_PATHS violations (no kernel/relayflowd/src/engine/hn_poller.rs)
-
-## Why this is the right work package
-
-Per ops/TARGET.md: "Gate 3 per RFC-0001 is the Garden: flows that build and improve other flows. Do the SMALLEST honest first step, not the whole thing. Good candidate: a flow that reads ops/BACKLOG.md, picks one entry by a deterministic rule, and emits a structured work package."
-
-This work package:
-- Stays strictly within gate 3 scope (SOFTWARE GARDEN foundation)
-- Is a CODE task as required
-- Does the smallest honest first step
-- Demonstrates "flows that build and improve other flows" (reading a backlog is the first step toward self-proposing work)
-- Has a clear, testable definition of done
-- Avoids all CONSTRAINTS from ops/TARGET.md (no server.rs, no hn-poller files)
-
-**ONE cycle, ten minutes.**
+- `sdk/src/` (new consumer code)
+- `sdk/tests/` (new consumer tests)
+- NO changes to `kernel/` (PR #19 is open)
+- NO changes to `sdk/src/demo-hn-monitor.ts` (PR #19 is open)
+
+## Definition of done
+
+1. **SDK code exists** that consumes an emitted work package
+2. **Validation tests exist** for:
+   - Missing title → typed refusal
+   - Empty title → typed refusal
+   - Missing scope → typed refusal
+   - Empty scope → typed refusal
+   - Missing definition of done → typed refusal
+   - Empty definition of done → typed refusal
+   - Valid package → accepted
+3. **Every new test is confirmed to FAIL against current code** with literal output pasted
+4. **SDK test suite passes:**
+   ```
+   cd sdk && npm test
+   ```
+   Paste the literal output showing all tests pass, 0 failed
+5. **Final verification** — as the LAST action, run:
+   ```
+   git status --porcelain
+   ```
+   And paste the output to make lost writes visible immediately
+
+## Out of scope
+
+- Kernel changes (different PR)
+- Integration with hn-monitor (different PR)
+- Any changes to the backlog picker itself (already merged in PRs #20, #21, #22)
+- Changes to flow execution or scheduling

sdk/src/index.ts

@@ -113,6 +113,13 @@ export { JOURNAL_WRITE_FAILED, PROTOCOL_VERSION } from './protocol.js';
 
 export { JournalClient, type JournalClientOptions } from './journal-client.js';
 
+export {
+  consumeWorkPackage,
+  type EmittedWorkPackage,
+  type WorkPackageConsumption,
+  type WorkPackageRefusalReason,
+} from './work-package-consumer.js';
+
 // Hacker News adapter — deliberately outside kernel/ (see sdk/src/hn-poller.ts).
 export {
   pollHackerNewsOnce,

sdk/src/work-package-consumer.ts

@@ -0,0 +1,46 @@
+/** The work-package shape emitted at the SDK boundary. */
+export interface EmittedWorkPackage {
+  title: string;
+  files_in_scope: string[];
+  definition_of_done: string[];
+  description?: string;
+  gate?: number | null;
+}
+
+export type WorkPackageRefusalReason =
+  | 'missing_title'
+  | 'missing_scope'
+  | 'missing_definition_of_done';
+
+export type WorkPackageConsumption =
+  | { accepted: true; work: EmittedWorkPackage }
+  | { accepted: false; reason: WorkPackageRefusalReason };
+
+/**
+ * Validate an emitted package before admitting it as runnable work.
+ * Refusals are data so callers must handle an unverifiable package explicitly.
+ */
+export function consumeWorkPackage(input: unknown): WorkPackageConsumption {
+  if (!isRecord(input) || !isNonEmptyString(input['title'])) {
+    return { accepted: false, reason: 'missing_title' };
+  }
+  if (!isNonEmptyStringArray(input['files_in_scope'])) {
+    return { accepted: false, reason: 'missing_scope' };
+  }
+  if (!isNonEmptyStringArray(input['definition_of_done'])) {
+    return { accepted: false, reason: 'missing_definition_of_done' };
+  }
+  return { accepted: true, work: input as unknown as EmittedWorkPackage };
+}
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+  return typeof value === 'object' && value !== null && !Array.isArray(value);
+}
+
+function isNonEmptyString(value: unknown): value is string {
+  return typeof value === 'string' && value.trim().length > 0;
+}
+
+function isNonEmptyStringArray(value: unknown): value is string[] {
+  return Array.isArray(value) && value.length > 0 && value.every(isNonEmptyString);
+}

sdk/tests/work-package-consumer.test.ts

@@ -0,0 +1,108 @@
+import { describe, expect, it } from 'vitest';
+import { consumeWorkPackage } from '../src/work-package-consumer.js';
+
+const validPackage = {
+  title: 'Build the work package consumer',
+  files_in_scope: ['sdk/src/', 'sdk/tests/'],
+  definition_of_done: ['cd sdk && npm test'],
+};
+
+async function consume(input: unknown) {
+  const module = await import('../src/work-package-consumer.js');
+  return module.consumeWorkPackage(input);
+}
+
+describe('work package consumer', () => {
+  it('refuses a missing title with a typed reason', async () => {
+    const { title: _, ...input } = validPackage;
+    expect(await consume(input)).toEqual({ accepted: false, reason: 'missing_title' });
+  });
+
+  it('refuses an empty title with a typed reason', async () => {
+    expect(await consume({ ...validPackage, title: '   ' })).toEqual({
+      accepted: false,
+      reason: 'missing_title',
+    });
+  });
+
+  it('refuses a missing scope with a typed reason', async () => {
+    const { files_in_scope: _, ...input } = validPackage;
+    expect(await consume(input)).toEqual({ accepted: false, reason: 'missing_scope' });
+  });
+
+  it('refuses an empty scope with a typed reason', async () => {
+    expect(await consume({ ...validPackage, files_in_scope: [] })).toEqual({
+      accepted: false,
+      reason: 'missing_scope',
+    });
+  });
+
+  it('refuses a missing definition of done with a typed reason', async () => {
+    const { definition_of_done: _, ...input } = validPackage;
+    expect(await consume(input)).toEqual({
+      accepted: false,
+      reason: 'missing_definition_of_done',
+    });
+  });
+
+  it('refuses an empty definition of done with a typed reason', async () => {
+    expect(await consume({ ...validPackage, definition_of_done: [] })).toEqual({
+      accepted: false,
+      reason: 'missing_definition_of_done',
+    });
+  });
+
+  it('accepts a valid package as runnable work', async () => {
+    expect(await consume(validPackage)).toEqual({ accepted: true, work: validPackage });
+  });
+});
+
+describe('the Garden join: picker output feeds the consumer', () => {
+  it('accepts a package the picker actually emits, and refuses one lacking a definition of done', () => {
+    // Review found the two halves did not fit (PR #23, P1): the picker emitted
+    // {title, description, files_in_scope, gate} and the consumer required
+    // definition_of_done, which the picker never produced — so the consumer
+    // would have refused EVERY real package and the loop could never accept
+    // anything. This test runs the flow's actual emit-package command so the
+    // shapes cannot drift apart again without failing here.
+    const { execFileSync } = require('node:child_process') as typeof import('node:child_process');
+    const { mkdtempSync, mkdirSync, writeFileSync, rmSync, readFileSync } = require('node:fs') as typeof import('node:fs');
+    const { tmpdir } = require('node:os') as typeof import('node:os');
+    const { join } = require('node:path') as typeof import('node:path');
+    const { load } = require('js-yaml') as typeof import('js-yaml');
+
+    const flowPath = join(__dirname, '..', '..', 'testdata', 'backlog-picker.flow.yaml');
+    const flow = load(readFileSync(flowPath, 'utf8')) as { steps: Array<{ id: string; command: string }> };
+    const step = (id: string) => flow.steps.find((s) => s.id === id)!.command;
+    const run = (cmd: string, cwd: string) => execFileSync('sh', ['-c', cmd], { cwd, encoding: 'utf8' });
+
+    const dir = mkdtempSync(join(tmpdir(), 'garden-join-'));
+    try {
+      mkdirSync(join(dir, 'ops'), { recursive: true });
+
+      // An entry carrying a runnable command: that IS its definition of done.
+      writeFileSync(
+        join(dir, 'ops', 'BACKLOG.md'),
+        '# Backlog\n\n- **Actionable entry** touches `sdk/src/x.ts`, verified by `npm test --silent`\n',
+      );
+      run(step('read-backlog'), dir);
+      run(step('select-entry'), dir);
+      const accepted = JSON.parse(run(step('emit-package'), dir));
+      expect(consumeWorkPackage(accepted).accepted, JSON.stringify(accepted)).toBe(true);
+
+      // An entry with no command names no way to verify itself.
+      writeFileSync(
+        join(dir, 'ops', 'BACKLOG.md'),
+        '# Backlog\n\n- **Scoped but unverifiable** touches `sdk/src/y.ts` but names no command\n',
+      );
+      run(step('read-backlog'), dir);
+      run(step('select-entry'), dir);
+      const refused = JSON.parse(run(step('emit-package'), dir));
+      const verdict = consumeWorkPackage(refused);
+      expect(verdict.accepted).toBe(false);
+      expect(verdict.accepted === false && verdict.reason).toBe('missing_definition_of_done');
+    } finally {
+      rmSync(dir, { recursive: true, force: true });
+    }
+  });
+});

testdata/backlog-picker.flow.yaml

@@ -16,4 +16,4 @@ steps:
     type: deterministic
     dependsOn: [select-entry]
     command: >-
-      node -e 'const fs=require("node:fs");const entry=JSON.parse(fs.readFileSync(".relayflow/backlog-picker-entry.json","utf8"));const text=entry.title+" "+entry.body;const files=[...new Set([...text.matchAll(/`([A-Za-z_][A-Za-z0-9._-]*(?:\/[A-Za-z0-9._-]*)+)`/g)].map(match=>match[1]).filter(candidate=>!/^\/|\/\//.test(candidate)))];const gate=text.match(/\bgate[ -]?(\d+)\b/i);process.stdout.write(JSON.stringify({title:entry.title,description:entry.body,files_in_scope:files,gate:gate?Number(gate[1]):null}))'
+      node -e 'const fs=require("node:fs");const entry=JSON.parse(fs.readFileSync(".relayflow/backlog-picker-entry.json","utf8"));const text=entry.title+" "+entry.body;const files=[...new Set([...text.matchAll(/`([A-Za-z_][A-Za-z0-9._-]*(?:\/[A-Za-z0-9._-]*)+)`/g)].map(match=>match[1]).filter(candidate=>!/^\/|\/\//.test(candidate)))];const gate=text.match(/\bgate[ -]?(\d+)\b/i);process.stdout.write(JSON.stringify({title:entry.title,description:entry.body,files_in_scope:files,gate:gate?Number(gate[1]):null,definition_of_done:(entry.body.match(/`[^`]+`/g)||[]).map(c=>c.slice(1,-1)).filter(c=>/\s/.test(c))}))'

testdata/backlog-picker.spec.canonical.json

@@ -1 +1 @@
-{"description":"Read ops/BACKLOG.md and deterministically emit its first work package.","name":"backlog-picker","steps":[{"command":"mkdir -p .relayflow && cat ops/BACKLOG.md > .relayflow/backlog-picker-source.md","depends_on":[],"id":"read-backlog","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}},{"command":"node -e 'const fs=require(\"node:fs\");fs.rmSync(\".relayflow/backlog-picker-entry.json\",{force:true});const text=fs.readFileSync(\".relayflow/backlog-picker-source.md\",\"utf8\");const match=text.match(/^- \\*\\*(.+?)\\*\\*\\s*(.*(?:\\n  .*)*)/m);if(!match)process.exit(1);const entry={title:match[1],body:match[2].replace(/\\s+/g,\" \").trim()};fs.writeFileSync(\".relayflow/backlog-picker-entry.json\",JSON.stringify(entry));process.stdout.write(JSON.stringify(entry))'","depends_on":["read-backlog"],"id":"select-entry","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}},{"command":"node -e 'const fs=require(\"node:fs\");const entry=JSON.parse(fs.readFileSync(\".relayflow/backlog-picker-entry.json\",\"utf8\"));const text=entry.title+\" \"+entry.body;const files=[...new Set([...text.matchAll(/`([A-Za-z_][A-Za-z0-9._-]*(?:\\/[A-Za-z0-9._-]*)+)`/g)].map(match=>match[1]).filter(candidate=>!/^\\/|\\/\\//.test(candidate)))];const gate=text.match(/\\bgate[ -]?(\\d+)\\b/i);process.stdout.write(JSON.stringify({title:entry.title,description:entry.body,files_in_scope:files,gate:gate?Number(gate[1]):null}))'","depends_on":["select-entry"],"id":"emit-package","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}}],"version":"0.1.0"}
+{"description":"Read ops/BACKLOG.md and deterministically emit its first work package.","name":"backlog-picker","steps":[{"command":"mkdir -p .relayflow && cat ops/BACKLOG.md > .relayflow/backlog-picker-source.md","depends_on":[],"id":"read-backlog","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}},{"command":"node -e 'const fs=require(\"node:fs\");fs.rmSync(\".relayflow/backlog-picker-entry.json\",{force:true});const text=fs.readFileSync(\".relayflow/backlog-picker-source.md\",\"utf8\");const match=text.match(/^- \\*\\*(.+?)\\*\\*\\s*(.*(?:\\n  .*)*)/m);if(!match)process.exit(1);const entry={title:match[1],body:match[2].replace(/\\s+/g,\" \").trim()};fs.writeFileSync(\".relayflow/backlog-picker-entry.json\",JSON.stringify(entry));process.stdout.write(JSON.stringify(entry))'","depends_on":["read-backlog"],"id":"select-entry","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}},{"command":"node -e 'const fs=require(\"node:fs\");const entry=JSON.parse(fs.readFileSync(\".relayflow/backlog-picker-entry.json\",\"utf8\"));const text=entry.title+\" \"+entry.body;const files=[...new Set([...text.matchAll(/`([A-Za-z_][A-Za-z0-9._-]*(?:\\/[A-Za-z0-9._-]*)+)`/g)].map(match=>match[1]).filter(candidate=>!/^\\/|\\/\\//.test(candidate)))];const gate=text.match(/\\bgate[ -]?(\\d+)\\b/i);process.stdout.write(JSON.stringify({title:entry.title,description:entry.body,files_in_scope:files,gate:gate?Number(gate[1]):null,definition_of_done:(entry.body.match(/`[^`]+`/g)||[]).map(c=>c.slice(1,-1)).filter(c=>/\\s/.test(c))}))'","depends_on":["select-entry"],"id":"emit-package","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}}],"version":"0.1.0"}