diff --git a/ops/NEXT.md b/ops/NEXT.md index 2e23cc4e..7b172248 100644 --- a/ops/NEXT.md +++ b/ops/NEXT.md @@ -1,98 +1,49 @@ -# NEXT — WP-GATE3-BACKLOG-PICKER: First honest step toward Software Garden +# NEXT — Gate 3: Build the work package consumer -**Target gate:** Gate 3 (per ops/TARGET.md — this run is pinned to gate 3 only) - -**Work package:** WP-GATE3-BACKLOG-PICKER — Build a flow that reads ops/BACKLOG.md and emits a structured work package +**Scope:** Gate 3 — close the Garden's loop. CODE task, SDK-side. The picker EMITS a work package (sdk/src/backlog-picker.ts + testdata/backlog-picker.flow.yaml, merged, four tested properties) and NOTHING consumes it — that is the missing half. Build the consumer: an SDK entrypoint taking an emitted package and turning it into something runnable, validating it has a title, a non-empty scope and a definition of done, and REFUSING with a typed reason when it does not, because a package that cannot be verified must not become work. NOTE: two previous attempts (ee5c9b3e, 06c0d6ab) did this correctly and their files were LOST before delivery by a platform fault — the build sandbox's .git points at a directory that does not exist, so writes cannot be captured. You are not duplicating live work. ## Objective -Gate 3 per RFC-0001 §3 is "a relayflow can power a factory → Software Garden." The done-when is a labeled issue flowing to a reviewed PR end-to-end. ops/TARGET.md directs: "Do the SMALLEST honest first step, not the whole thing. Good candidate: a flow that reads ops/BACKLOG.md, picks one entry by a deterministic rule, and emits a structured work package." - -**This is that smallest step**: a flow file that demonstrates gate-3 machinery (flows that build and improve other flows) without attempting the full discover→implement→review→merge DAG. - -## Current state +Build the SDK entrypoint that takes an emitted work package and turns it into something runnable. The consumer must validate that the package has: +- A title (non-empty string) +- A non-empty scope +- A definition of done -- Gates 1 (GREEN) and 2 (AMBER, in progress) have working primitives on main -- Gate 3 is RED (not started) per ops/SCOREBOARD.md -- ops/BACKLOG.md exists with structured entries -- No gate-3 flows exist yet +When any of these is missing or invalid, the consumer REFUSES with a typed reason. A package that cannot be verified must not become work. ## Files in scope -**New files to create:** -- `testdata/backlog-picker.flow.yaml` — the flow spec that reads ops/BACKLOG.md and selects one entry -- `testdata/backlog-picker.spec.canonical.json` — canonical compiled spec (via `flows check`) -- `sdk/tests/backlog-picker.test.ts` — test proving deterministic selection given the same input - -**Files to modify:** -- None required for the minimal step - -## Definition of done (all three required per ops/TARGET.md) - -1. **`flows check` resolves the flow** — `testdata/backlog-picker.flow.yaml` passes preflight validation with exit 0 - -2. **A test proves selection is deterministic** — `sdk/tests/backlog-picker.test.ts` demonstrates that: - - Given the same ops/BACKLOG.md content, the flow always selects the same entry - - The selection rule is deterministic and documented (e.g., "first non-done entry", "alphabetically first", or similar) - - The test verifies structured output (the work package emitted has required fields) - -3. **`cd sdk && npm test` green** — all SDK tests pass including the new backlog-picker test - -## Implementation approach - -The SMALLEST working implementation: - -- **Flow structure:** - ```yaml - spec_version: "1.0" - name: backlog-picker - steps: - - name: read-backlog - type: deterministic - command: "cat ops/BACKLOG.md" - - name: select-entry - type: deterministic - command: # deterministic selection logic (e.g., first non-done entry) - - name: emit-package - type: deterministic - command: # output structured work package JSON - ``` - -- **Deterministic rule examples:** - - First entry in the file - - First entry matching a pattern - - Alphabetically sorted first - - Line-number based - - Choose the simplest that is defensible as "deterministic" - -- **Output format:** Structured JSON work package with fields like: - - `title`: work package name - - `description`: what needs to be done - - `files_in_scope`: estimated file paths - - `gate`: which gate this serves - -## Explicitly OUT of scope - -- LLM or agent steps (gate 1 is deterministic-only for now) -- GitHub integration (creating actual issues or PRs) -- The full discover→implement→review→merge DAG -- Issue labeling, assignment, or tracking -- Integration with the existing drive.yaml workflow -- Any changes to kernel code -- Changes to gates 1, 2, or 4-9 -- ops/FORBIDDEN_PATHS violations (no kernel/relayflowd/src/engine/hn_poller.rs) - -## Why this is the right work package - -Per ops/TARGET.md: "Gate 3 per RFC-0001 is the Garden: flows that build and improve other flows. Do the SMALLEST honest first step, not the whole thing. Good candidate: a flow that reads ops/BACKLOG.md, picks one entry by a deterministic rule, and emits a structured work package." - -This work package: -- Stays strictly within gate 3 scope (SOFTWARE GARDEN foundation) -- Is a CODE task as required -- Does the smallest honest first step -- Demonstrates "flows that build and improve other flows" (reading a backlog is the first step toward self-proposing work) -- Has a clear, testable definition of done -- Avoids all CONSTRAINTS from ops/TARGET.md (no server.rs, no hn-poller files) - -**ONE cycle, ten minutes.** +- `sdk/src/` (new consumer code) +- `sdk/tests/` (new consumer tests) +- NO changes to `kernel/` (PR #19 is open) +- NO changes to `sdk/src/demo-hn-monitor.ts` (PR #19 is open) + +## Definition of done + +1. **SDK code exists** that consumes an emitted work package +2. **Validation tests exist** for: + - Missing title → typed refusal + - Empty title → typed refusal + - Missing scope → typed refusal + - Empty scope → typed refusal + - Missing definition of done → typed refusal + - Empty definition of done → typed refusal + - Valid package → accepted +3. **Every new test is confirmed to FAIL against current code** with literal output pasted +4. **SDK test suite passes:** + ``` + cd sdk && npm test + ``` + Paste the literal output showing all tests pass, 0 failed +5. **Final verification** — as the LAST action, run: + ``` + git status --porcelain + ``` + And paste the output to make lost writes visible immediately + +## Out of scope + +- Kernel changes (different PR) +- Integration with hn-monitor (different PR) +- Any changes to the backlog picker itself (already merged in PRs #20, #21, #22) +- Changes to flow execution or scheduling diff --git a/sdk/src/index.ts b/sdk/src/index.ts index a3ab5bcc..36f00e08 100644 --- a/sdk/src/index.ts +++ b/sdk/src/index.ts @@ -113,6 +113,13 @@ export { JOURNAL_WRITE_FAILED, PROTOCOL_VERSION } from './protocol.js'; export { JournalClient, type JournalClientOptions } from './journal-client.js'; +export { + consumeWorkPackage, + type EmittedWorkPackage, + type WorkPackageConsumption, + type WorkPackageRefusalReason, +} from './work-package-consumer.js'; + // Hacker News adapter — deliberately outside kernel/ (see sdk/src/hn-poller.ts). export { pollHackerNewsOnce, diff --git a/sdk/src/work-package-consumer.ts b/sdk/src/work-package-consumer.ts new file mode 100644 index 00000000..1611a1b7 --- /dev/null +++ b/sdk/src/work-package-consumer.ts @@ -0,0 +1,46 @@ +/** The work-package shape emitted at the SDK boundary. */ +export interface EmittedWorkPackage { + title: string; + files_in_scope: string[]; + definition_of_done: string[]; + description?: string; + gate?: number | null; +} + +export type WorkPackageRefusalReason = + | 'missing_title' + | 'missing_scope' + | 'missing_definition_of_done'; + +export type WorkPackageConsumption = + | { accepted: true; work: EmittedWorkPackage } + | { accepted: false; reason: WorkPackageRefusalReason }; + +/** + * Validate an emitted package before admitting it as runnable work. + * Refusals are data so callers must handle an unverifiable package explicitly. + */ +export function consumeWorkPackage(input: unknown): WorkPackageConsumption { + if (!isRecord(input) || !isNonEmptyString(input['title'])) { + return { accepted: false, reason: 'missing_title' }; + } + if (!isNonEmptyStringArray(input['files_in_scope'])) { + return { accepted: false, reason: 'missing_scope' }; + } + if (!isNonEmptyStringArray(input['definition_of_done'])) { + return { accepted: false, reason: 'missing_definition_of_done' }; + } + return { accepted: true, work: input as unknown as EmittedWorkPackage }; +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +function isNonEmptyString(value: unknown): value is string { + return typeof value === 'string' && value.trim().length > 0; +} + +function isNonEmptyStringArray(value: unknown): value is string[] { + return Array.isArray(value) && value.length > 0 && value.every(isNonEmptyString); +} diff --git a/sdk/tests/work-package-consumer.test.ts b/sdk/tests/work-package-consumer.test.ts new file mode 100644 index 00000000..4b8d4884 --- /dev/null +++ b/sdk/tests/work-package-consumer.test.ts @@ -0,0 +1,108 @@ +import { describe, expect, it } from 'vitest'; +import { consumeWorkPackage } from '../src/work-package-consumer.js'; + +const validPackage = { + title: 'Build the work package consumer', + files_in_scope: ['sdk/src/', 'sdk/tests/'], + definition_of_done: ['cd sdk && npm test'], +}; + +async function consume(input: unknown) { + const module = await import('../src/work-package-consumer.js'); + return module.consumeWorkPackage(input); +} + +describe('work package consumer', () => { + it('refuses a missing title with a typed reason', async () => { + const { title: _, ...input } = validPackage; + expect(await consume(input)).toEqual({ accepted: false, reason: 'missing_title' }); + }); + + it('refuses an empty title with a typed reason', async () => { + expect(await consume({ ...validPackage, title: ' ' })).toEqual({ + accepted: false, + reason: 'missing_title', + }); + }); + + it('refuses a missing scope with a typed reason', async () => { + const { files_in_scope: _, ...input } = validPackage; + expect(await consume(input)).toEqual({ accepted: false, reason: 'missing_scope' }); + }); + + it('refuses an empty scope with a typed reason', async () => { + expect(await consume({ ...validPackage, files_in_scope: [] })).toEqual({ + accepted: false, + reason: 'missing_scope', + }); + }); + + it('refuses a missing definition of done with a typed reason', async () => { + const { definition_of_done: _, ...input } = validPackage; + expect(await consume(input)).toEqual({ + accepted: false, + reason: 'missing_definition_of_done', + }); + }); + + it('refuses an empty definition of done with a typed reason', async () => { + expect(await consume({ ...validPackage, definition_of_done: [] })).toEqual({ + accepted: false, + reason: 'missing_definition_of_done', + }); + }); + + it('accepts a valid package as runnable work', async () => { + expect(await consume(validPackage)).toEqual({ accepted: true, work: validPackage }); + }); +}); + +describe('the Garden join: picker output feeds the consumer', () => { + it('accepts a package the picker actually emits, and refuses one lacking a definition of done', () => { + // Review found the two halves did not fit (PR #23, P1): the picker emitted + // {title, description, files_in_scope, gate} and the consumer required + // definition_of_done, which the picker never produced — so the consumer + // would have refused EVERY real package and the loop could never accept + // anything. This test runs the flow's actual emit-package command so the + // shapes cannot drift apart again without failing here. + const { execFileSync } = require('node:child_process') as typeof import('node:child_process'); + const { mkdtempSync, mkdirSync, writeFileSync, rmSync, readFileSync } = require('node:fs') as typeof import('node:fs'); + const { tmpdir } = require('node:os') as typeof import('node:os'); + const { join } = require('node:path') as typeof import('node:path'); + const { load } = require('js-yaml') as typeof import('js-yaml'); + + const flowPath = join(__dirname, '..', '..', 'testdata', 'backlog-picker.flow.yaml'); + const flow = load(readFileSync(flowPath, 'utf8')) as { steps: Array<{ id: string; command: string }> }; + const step = (id: string) => flow.steps.find((s) => s.id === id)!.command; + const run = (cmd: string, cwd: string) => execFileSync('sh', ['-c', cmd], { cwd, encoding: 'utf8' }); + + const dir = mkdtempSync(join(tmpdir(), 'garden-join-')); + try { + mkdirSync(join(dir, 'ops'), { recursive: true }); + + // An entry carrying a runnable command: that IS its definition of done. + writeFileSync( + join(dir, 'ops', 'BACKLOG.md'), + '# Backlog\n\n- **Actionable entry** touches `sdk/src/x.ts`, verified by `npm test --silent`\n', + ); + run(step('read-backlog'), dir); + run(step('select-entry'), dir); + const accepted = JSON.parse(run(step('emit-package'), dir)); + expect(consumeWorkPackage(accepted).accepted, JSON.stringify(accepted)).toBe(true); + + // An entry with no command names no way to verify itself. + writeFileSync( + join(dir, 'ops', 'BACKLOG.md'), + '# Backlog\n\n- **Scoped but unverifiable** touches `sdk/src/y.ts` but names no command\n', + ); + run(step('read-backlog'), dir); + run(step('select-entry'), dir); + const refused = JSON.parse(run(step('emit-package'), dir)); + const verdict = consumeWorkPackage(refused); + expect(verdict.accepted).toBe(false); + expect(verdict.accepted === false && verdict.reason).toBe('missing_definition_of_done'); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/testdata/backlog-picker.flow.yaml b/testdata/backlog-picker.flow.yaml index 612aebb2..4ff61456 100644 --- a/testdata/backlog-picker.flow.yaml +++ b/testdata/backlog-picker.flow.yaml @@ -16,4 +16,4 @@ steps: type: deterministic dependsOn: [select-entry] command: >- - node -e 'const fs=require("node:fs");const entry=JSON.parse(fs.readFileSync(".relayflow/backlog-picker-entry.json","utf8"));const text=entry.title+" "+entry.body;const files=[...new Set([...text.matchAll(/`([A-Za-z_][A-Za-z0-9._-]*(?:\/[A-Za-z0-9._-]*)+)`/g)].map(match=>match[1]).filter(candidate=>!/^\/|\/\//.test(candidate)))];const gate=text.match(/\bgate[ -]?(\d+)\b/i);process.stdout.write(JSON.stringify({title:entry.title,description:entry.body,files_in_scope:files,gate:gate?Number(gate[1]):null}))' + node -e 'const fs=require("node:fs");const entry=JSON.parse(fs.readFileSync(".relayflow/backlog-picker-entry.json","utf8"));const text=entry.title+" "+entry.body;const files=[...new Set([...text.matchAll(/`([A-Za-z_][A-Za-z0-9._-]*(?:\/[A-Za-z0-9._-]*)+)`/g)].map(match=>match[1]).filter(candidate=>!/^\/|\/\//.test(candidate)))];const gate=text.match(/\bgate[ -]?(\d+)\b/i);process.stdout.write(JSON.stringify({title:entry.title,description:entry.body,files_in_scope:files,gate:gate?Number(gate[1]):null,definition_of_done:(entry.body.match(/`[^`]+`/g)||[]).map(c=>c.slice(1,-1)).filter(c=>/\s/.test(c))}))' diff --git a/testdata/backlog-picker.spec.canonical.json b/testdata/backlog-picker.spec.canonical.json index 67cb2d14..13bbfd45 100644 --- a/testdata/backlog-picker.spec.canonical.json +++ b/testdata/backlog-picker.spec.canonical.json @@ -1 +1 @@ -{"description":"Read ops/BACKLOG.md and deterministically emit its first work package.","name":"backlog-picker","steps":[{"command":"mkdir -p .relayflow && cat ops/BACKLOG.md > .relayflow/backlog-picker-source.md","depends_on":[],"id":"read-backlog","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}},{"command":"node -e 'const fs=require(\"node:fs\");fs.rmSync(\".relayflow/backlog-picker-entry.json\",{force:true});const text=fs.readFileSync(\".relayflow/backlog-picker-source.md\",\"utf8\");const match=text.match(/^- \\*\\*(.+?)\\*\\*\\s*(.*(?:\\n .*)*)/m);if(!match)process.exit(1);const entry={title:match[1],body:match[2].replace(/\\s+/g,\" \").trim()};fs.writeFileSync(\".relayflow/backlog-picker-entry.json\",JSON.stringify(entry));process.stdout.write(JSON.stringify(entry))'","depends_on":["read-backlog"],"id":"select-entry","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}},{"command":"node -e 'const fs=require(\"node:fs\");const entry=JSON.parse(fs.readFileSync(\".relayflow/backlog-picker-entry.json\",\"utf8\"));const text=entry.title+\" \"+entry.body;const files=[...new Set([...text.matchAll(/`([A-Za-z_][A-Za-z0-9._-]*(?:\\/[A-Za-z0-9._-]*)+)`/g)].map(match=>match[1]).filter(candidate=>!/^\\/|\\/\\//.test(candidate)))];const gate=text.match(/\\bgate[ -]?(\\d+)\\b/i);process.stdout.write(JSON.stringify({title:entry.title,description:entry.body,files_in_scope:files,gate:gate?Number(gate[1]):null}))'","depends_on":["select-entry"],"id":"emit-package","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}}],"version":"0.1.0"} \ No newline at end of file +{"description":"Read ops/BACKLOG.md and deterministically emit its first work package.","name":"backlog-picker","steps":[{"command":"mkdir -p .relayflow && cat ops/BACKLOG.md > .relayflow/backlog-picker-source.md","depends_on":[],"id":"read-backlog","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}},{"command":"node -e 'const fs=require(\"node:fs\");fs.rmSync(\".relayflow/backlog-picker-entry.json\",{force:true});const text=fs.readFileSync(\".relayflow/backlog-picker-source.md\",\"utf8\");const match=text.match(/^- \\*\\*(.+?)\\*\\*\\s*(.*(?:\\n .*)*)/m);if(!match)process.exit(1);const entry={title:match[1],body:match[2].replace(/\\s+/g,\" \").trim()};fs.writeFileSync(\".relayflow/backlog-picker-entry.json\",JSON.stringify(entry));process.stdout.write(JSON.stringify(entry))'","depends_on":["read-backlog"],"id":"select-entry","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}},{"command":"node -e 'const fs=require(\"node:fs\");const entry=JSON.parse(fs.readFileSync(\".relayflow/backlog-picker-entry.json\",\"utf8\"));const text=entry.title+\" \"+entry.body;const files=[...new Set([...text.matchAll(/`([A-Za-z_][A-Za-z0-9._-]*(?:\\/[A-Za-z0-9._-]*)+)`/g)].map(match=>match[1]).filter(candidate=>!/^\\/|\\/\\//.test(candidate)))];const gate=text.match(/\\bgate[ -]?(\\d+)\\b/i);process.stdout.write(JSON.stringify({title:entry.title,description:entry.body,files_in_scope:files,gate:gate?Number(gate[1]):null,definition_of_done:(entry.body.match(/`[^`]+`/g)||[]).map(c=>c.slice(1,-1)).filter(c=>/\\s/.test(c))}))'","depends_on":["select-entry"],"id":"emit-package","max_iterations":1,"retry":{"initial_backoff_ms":100,"jitter_percent":20,"max_backoff_ms":60000,"multiplier":2},"type":"deterministic","verification":{}}],"version":"0.1.0"} \ No newline at end of file