diff --git a/.agents/skills/improving-by-trial/SKILL.md b/.agents/skills/improving-by-trial/SKILL.md new file mode 100644 index 0000000..7570515 --- /dev/null +++ b/.agents/skills/improving-by-trial/SKILL.md @@ -0,0 +1,120 @@ +--- +name: improving-by-trial +description: Improve this repository's plugins by trial — run fresh, isolated agents that get only a user's sentence and this repository's link, collect where they got stuck, fix what belongs here, file what belongs elsewhere, and repeat until the trial passes. Use when asked to trial, dogfood, test the onboarding or a plugin end to end, run a fresh agent against the plugins, or iterate on trial findings before a release. +--- + +# Improving the plugins by trial + +A trial is a fresh agent with nothing but a user's sentence and a link. It shows what the skills +fail to say. The gate cannot show that: every check passes while an agent still guesses. + +## 1. Build a sandbox from this checkout + +```console +task trial:sandbox NAME= +``` + +This creates `/var/tmp/b10x-trials-$USER//` (the Taskfile's `TRIALS`) with its own `home/`, a `b10x` built from this +checkout, and an `env` file. The `env` file sets `HOME` to the sandbox and `B10X_MARKETPLACE` to +this checkout, so `b10x init` installs the plugins as they are in the working tree, before any +release. It also unsets `ANTHROPIC_API_KEY`, `TMPDIR` and `TMPPREFIX`. Put the fixture the trial +needs (a small service, a `TODO.md`, an OpenAPI document) under `work/`, and commit it there with +`git` when the trial needs history or a remote. + +To trial the released version instead, remove `home/.local/bin/b10x` and unset `B10X_MARKETPLACE` +in `env`; the agent then follows `SETUP.md` from the release. + +## 2. Run it headless, never as a sub-agent + +```console +task trial:run NAME= PROMPT='' [DIR=] +``` + +The task copies the operator's credentials in (mode 600), runs `claude -p` with the sandbox's +`env`, `--strict-mcp-config` (no MCP server, including the account's claude.ai connectors) and +stream-json output into `run.jsonl`, and deletes the credentials when it finishes. + +**Never run a trial as a sub-agent of the working session.** A sub-agent gets the session's own +agents and plugins whatever `HOME` its shell uses. In trial 3 the aep critics that ran were this +machine's, not the plugin's under test. + +The prompt is what a user would type, plus the link when the trial is about finding the +repository. Ask the agent to end by listing the skills and agents it used, quoting any text that +confused it, and pasting the final command output verbatim. Give it no other hints. + +## 3. Only an isolated run counts + +`trial:run` ends with: + +```console +agentplugins-check trial-isolation /run.jsonl --sandbox --version +``` + +It fails when a plugin loaded from outside the sandbox home and the checkout, when a plugin is not +in the sandbox's own registry as it was before the run, when a plugin is not at the version under +test, or when an agent or skill came from a plugin the sandbox did not load. A failing run is +discarded, not interpreted. + +**A sandbox under the home directory is not isolated.** Claude Code reads `CLAUDE.md`, +`.claude/CLAUDE.md` and `.claude/settings*.json` in every directory above its working directory. In +trial 3, sandboxes under `~/.cache` loaded the operator's `~/.claude/CLAUDE.md` (one agent followed +its commit rules) and a `.claude/settings.local.json` an earlier trial left in `~/.cache`, which +enabled `aep@b10x`. `trial:sandbox` refuses to build below any of those files; that is why `TRIALS` +is outside `$HOME`. Neither leak shows in the `init` event, so the placement is the only guard. An upgrade trial seeds older plugins on +purpose: run it with `SEEDED=true`, so plugins must match what the sandbox was seeded with. + +## 4. Read the run + +From `run.jsonl`: + +| collect | how | +|---|---| +| tool calls | count of `tool_use` blocks | +| stuck points | every refusal and error in `tool_result`s, verbatim | +| guesses | what the final report says it inferred or could not find | +| waste | calls repeated, files read twice, commands that failed and were retried unchanged | +| the outcome | the verbatim `validate` / `generate` / plan output it pasted | + +Before writing a finding into a skill, reproduce each claimed behaviour with the released CLI. A +trial agent's explanation of a refusal is a hypothesis. + +## 5. Triage every finding to one owner + +| the cause | where the fix goes | +|---|---| +| a skill, agent, `SETUP.md`, the catalog or the `b10x` CLI | here: fix it on a branch, with a test for CLI behaviour | +| a product CLI or language (`ess`, `aep`, `worktree`) | an issue in that repository, created through the bot (`AGENTS.md`); the skill here documents the workaround until it is fixed | +| both | both: the workaround here, the issue there, each naming the other | + +The issue body has the trial name, the exact command and output, the expected behaviour, the +workaround the skill now documents, and ends with the line *Found by an agentplugins trial*. That +line is the ledger: the bot may not label issues (403), so the list is a body search. + +```console +gh search issues --owner beyond10x --state open --match body "Found by an agentplugins trial" +``` + +Check that list before each round, and delete a workaround from the skills once its issue is +fixed and released. + +## 6. Fix, gate, re-run + +1. Fix on a managed worktree branch. +2. Run `task check`, and `agentplugins-check tools` when a CLI command or the ESS example changed. +3. Rebuild the sandbox (`task trial:sandbox`) and re-run the trial that found the problem. It + passes when the finding does not recur. New findings go back to step 5. +4. Release when every trial of the round passes. + +Each round has 3 ESS trials (a new specification; a retrofit of an existing service; generate plus +a synthesized suite), and 1 trial each for aep planning, worktree, and onboarding or upgrade. +Change the domain each round so the agents cannot copy the previous answer from the skills. + +## 7. Clean up + +`trial:run` deletes the credentials it copied. When the round is released, check that no +credentials are left in any sandbox and remove the sandboxes: + +```console +ls /var/tmp/b10x-trials-$USER/*/home/.claude/.credentials.json +rm -rf /var/tmp/b10x-trials-$USER/ +``` diff --git a/.claude/skills/improving-by-trial b/.claude/skills/improving-by-trial new file mode 120000 index 0000000..4517d5b --- /dev/null +++ b/.claude/skills/improving-by-trial @@ -0,0 +1 @@ +../../.agents/skills/improving-by-trial \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md index ea78326..ae3fd87 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,6 +15,7 @@ CLIs. Serves O2 (decisions as data) and O3 (any harness). | `evals/` | eval corpus ([`evals/README.md`](evals/README.md)) | | `website/` | public docs; must pass `task site-build` | | `SETUP.md` | agent bootstrap, published as a release asset | +| `.agents/skills/improving-by-trial/` | how plugins are improved: isolated headless trials (`task trial:sandbox`, `task trial:run`), triage, fix, re-run | ## Rules @@ -27,6 +28,8 @@ CLIs. Serves O2 (decisions as data) and O3 (any harness). - Retired names appear only where the gate allows them (`CHANGELOG.md`, `changes/`, `.engineering/`, `catalog.json`, `crates/b10x/`, the checker's own table). - Anything executable is Rust. +- Trial findings for another repository become an issue there, ending *Found by an agentplugins + trial*; the skill here documents the workaround until it is fixed ([`improving-by-trial`](.agents/skills/improving-by-trial/SKILL.md)). ## Gate diff --git a/CHANGELOG.md b/CHANGELOG.md index bc553d9..38491a4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,51 @@ # Changelog +## [0.14.3] — 2026-09-25 + +From trial 3: five fresh agents on 0.14.2 (an ESS specification from nothing, an ESS retrofit of an +existing service, an ESS pipeline through generate and a synthesized suite, AEP planning over a +`TODO.md`, worktree onboarding) plus a migration of a seeded older install. + +- `ess:specifying`: lifecycle state names start upper-case; `ess-inputs.yaml` is documented and + recommended once anything is generated, since a directory without it is read whole; + `generate --out ` writes its own `schema/` or `openapi/` below ``; an entity invariant + needs a view holding every state and publishing the fields it reads, or `synthesize` refuses the + check. The syntax example gains that view: it synthesized with 3 refusals, now 12 scenarios and 0. +- The syntax reference says how to write rules the trials hit: a limit read from stored state, and + no double booking (a per-slot lifecycle, or `UNMAPPED:`). A time range is ordered by an `Integer` + length; comparing two `Timestamp`s validates and then fails `synthesize`, and `Duration` does not + compare with a number. +- `ess:retrofitting`: `ess infra import openapi` reads OpenAPI 3.1 only; `aep plan reverse openapi` + accepts 3.0. Its description named the retired `specify` and `coverage` skills. +- `ess:testing-conformance`: what the built-in targets `interpreted`, `oracle-fixture` and `billing` + are, why a new domain reports nothing useful against them, and `synthesize --target go|typescript` + to hold a real implementation to the suite. +- `aep:planning`: the `--protocols` and `--profile` values for `reverse init`; body files live in a + git-ignored directory of the repository, not `$TMPDIR`; a critic's empty findings fence is recorded as + `[]`, which the store accepts, since it refuses the empty fence. Two references to `ess skill`, which no longer exists, now name + `b10x skill ess:specifying`. +- `aep:migrating`: `reverse init` creates the store in every case; a backlog item that introduces a + new noun gets its ESS domain before its stories. Provenance says whose revisions it counts. +- `b10x:init` and `aep:init` offer `ess` with `aep`. +- The design and parallel-safety critics name the same two remedies for a shared file (an ordering + edge, or splitting the file) instead of asking for opposite changes in consecutive rounds. +- `worktree:init`: where to get a profile and where it goes; `doctor --check` exits 0 with + `profiles=0`; cleanup needs a remote. `worktree:managing-worktrees`: switch to a branch before the + first commit. +- `b10x:routing` routes ESS and worktree work to their skills instead of to `ess skill`. +- `b10x upgrade` installs the replacement of a legacy plugin it removes; it used to remove it and + only note the replacement as missing. A legacy `local` install is not reinstalled at `local` when + the user scope covers it. A plan names Beyond10x plugins on a host it does not cover. `b10x check` no longer says nothing is set up + when only a legacy plugin is installed. +- `agentplugins-check tools` also synthesizes and generates the ESS syntax example. +- `agentplugins-check` parses its arguments with clap; new `trial-isolation` checks that a headless + trial loaded only the plugins its sandbox installed, at the version under test, and no MCP server. + Sandboxes live outside `$HOME` (`/var/tmp/b10x-trials-$USER`): below it, a trial loaded the + operator's `CLAUDE.md` and a settings file an earlier trial left behind. `task trial:sandbox` and + `task trial:run`, and the repository skill `improving-by-trial`, describe the loop. +- The README lists every plugin's skills and agents, linked to their files; the gate keeps it + exact. The retired-name sweep covers `ess skill` and the `beyond10x/ess` marketplace. + ## [0.14.2] — 2026-09-24 From trial 2, in which fresh agents given only a sentence and this repository's link onboarded diff --git a/Cargo.lock b/Cargo.lock index 62a6c39..dbc6ac6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4,8 +4,9 @@ version = 4 [[package]] name = "agentplugins-check" -version = "0.14.2" +version = "0.14.3" dependencies = [ + "clap", "serde", "serde_json", "serde_yaml", @@ -75,7 +76,7 @@ dependencies = [ [[package]] name = "b10x" -version = "0.14.2" +version = "0.14.3" dependencies = [ "clap", "serde", diff --git a/Cargo.toml b/Cargo.toml index f600128..f6c82c3 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -3,7 +3,7 @@ resolver = "2" members = ["crates/agentplugins-check", "crates/b10x"] [workspace.package] -version = "0.14.2" +version = "0.14.3" edition = "2021" rust-version = "1.85" license = "Apache-2.0" diff --git a/README.md b/README.md index ed327d5..03b010f 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,23 @@ Codex: `codex plugin marketplace add beyond10x/agentplugins` — then install wh | [`connectors`](website/docs/plugins/connectors.md) | integrations | `/plugin install connectors@b10x` | `codex plugin add connectors@b10x` | | [`b10x`](website/docs/plugins/b10x.md) | setup, upgrades, routing | `/plugin install b10x@b10x` | `codex plugin add b10x@b10x` | +What each plugin ships: + + +- [`ess`](plugins/ess/) · [docs](website/docs/plugins/ess.md) + - skills: [`init`](plugins/ess/skills/init/SKILL.md) · [`upgrade`](plugins/ess/skills/upgrade/SKILL.md) · [`retrofitting`](plugins/ess/skills/retrofitting/SKILL.md) · [`specifying`](plugins/ess/skills/specifying/SKILL.md) · [`testing-conformance`](plugins/ess/skills/testing-conformance/SKILL.md) + - agents: [`author`](plugins/ess/agents/author.md) · [`conformance`](plugins/ess/agents/conformance.md) · [`retrofitter`](plugins/ess/agents/retrofitter.md) +- [`aep`](plugins/aep/) · [docs](website/docs/plugins/aep.md) + - skills: [`init`](plugins/aep/skills/init/SKILL.md) · [`upgrade`](plugins/aep/skills/upgrade/SKILL.md) · [`implementing`](plugins/aep/skills/implementing/SKILL.md) · [`migrating`](plugins/aep/skills/migrating/SKILL.md) · [`planning`](plugins/aep/skills/planning/SKILL.md) + - agents: [`adversary`](plugins/aep/agents/adversary.md) · [`decomposer`](plugins/aep/agents/decomposer.md) · [`implementor`](plugins/aep/agents/implementor.md) · [`plan-critic-acceptance`](plugins/aep/agents/plan-critic-acceptance.md) · [`plan-critic-design`](plugins/aep/agents/plan-critic-design.md) · [`plan-critic-parallel-safety`](plugins/aep/agents/plan-critic-parallel-safety.md) · [`plan-critic-scope`](plugins/aep/agents/plan-critic-scope.md) · [`plan-reviewer`](plugins/aep/agents/plan-reviewer.md) · [`reverse-engineer`](plugins/aep/agents/reverse-engineer.md) · [`security-reviewer`](plugins/aep/agents/security-reviewer.md) · [`story-scoper`](plugins/aep/agents/story-scoper.md) +- [`worktree`](plugins/worktree/) · [docs](website/docs/plugins/worktree.md) + - skills: [`init`](plugins/worktree/skills/init/SKILL.md) · [`upgrade`](plugins/worktree/skills/upgrade/SKILL.md) · [`managing-worktrees`](plugins/worktree/skills/managing-worktrees/SKILL.md) +- [`connectors`](plugins/connectors/) · [docs](website/docs/plugins/connectors.md) + - skills: [`init`](plugins/connectors/skills/init/SKILL.md) · [`upgrade`](plugins/connectors/skills/upgrade/SKILL.md) · [`integrating`](plugins/connectors/skills/integrating/SKILL.md) +- [`b10x`](plugins/b10x/) · [docs](website/docs/plugins/b10x.md) + - skills: [`init`](plugins/b10x/skills/init/SKILL.md) · [`upgrade`](plugins/b10x/skills/upgrade/SKILL.md) · [`authoring-plugins`](plugins/b10x/skills/authoring-plugins/SKILL.md) · [`routing`](plugins/b10x/skills/routing/SKILL.md) + + Most plugins drive a CLI of the same name. To have an agent install plugins and matching CLIs, and migrate older installs, tell it: *"Set up Beyond10x: follow https://github.com/beyond10x/agentplugins/releases/latest/download/SETUP.md"*. diff --git a/Taskfile.yml b/Taskfile.yml index c16ba0c..f0305fb 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -1,5 +1,10 @@ version: '3' +vars: + # Outside the home directory: Claude Code reads CLAUDE.md, .claude/CLAUDE.md and .claude/settings*.json + # in every directory above its working directory, so a sandbox under $HOME inherits the operator's. + TRIALS: '/var/tmp/b10x-trials-{{.USER}}' + tasks: check: desc: Run the complete offline repository gate. @@ -17,6 +22,71 @@ tasks: cmds: - cargo run --quiet --locked --bin agentplugins-check -- evals + trial:sandbox: + desc: "Create a clean trial sandbox for this checkout: task trial:sandbox NAME= (see .agents/skills/improving-by-trial)." + requires: + vars: [NAME] + vars: + T: '{{.TRIALS}}/{{.NAME}}' + cmds: + # Claude Code also reads `.claude/settings*.json` in the directories above its working + # directory; one there enables plugins in every sandbox below it. + - | + d='{{.T}}' + while [ "$d" != / ]; do + d="$(dirname "$d")" + for f in "$d/CLAUDE.md" "$d/.claude/CLAUDE.md" "$d/.claude/settings.json" "$d/.claude/settings.local.json"; do + if [ -e "$f" ]; then + echo "refusing: $f would load into every sandbox below it; set TRIALS elsewhere" + exit 1 + fi + done + done + - rm -rf '{{.T}}' + - mkdir -p -m 700 '{{.TRIALS}}' + - mkdir -p '{{.T}}/home/.local/bin' '{{.T}}/work' '{{.T}}/tools' + - for t in claude codex; do p="$(command -v $t)" && ln -sf "$(readlink -f "$p")" '{{.T}}/tools/'$t; done; true + - for t in cargo rustc rustup; do ln -sf "$HOME/.cargo/bin/$t" '{{.T}}/tools/'$t; done + - echo '{}' > '{{.T}}/home/.claude.json' + - nice -n 19 env CARGO_BUILD_JOBS=4 cargo build --quiet --locked -p b10x + - cp "${CARGO_TARGET_DIR:-target}/debug/b10x" '{{.T}}/home/.local/bin/b10x' + - | + { + echo "export HOME='{{.T}}/home'" + echo "export PATH='{{.T}}/home/.local/bin:{{.T}}/home/.cargo/bin:{{.T}}/tools:/usr/bin:/bin'" + echo "export RUSTUP_HOME='$HOME/.rustup' CARGO_HOME='$HOME/.cargo' CARGO_BUILD_JOBS=4" + echo "export B10X_MARKETPLACE='{{.ROOT_DIR}}'" + echo "export GIT_AUTHOR_NAME=trial GIT_AUTHOR_EMAIL=trial@example.invalid GIT_COMMITTER_NAME=trial GIT_COMMITTER_EMAIL=trial@example.invalid" + echo "unset ANTHROPIC_API_KEY TMPDIR TMPPREFIX" + } > '{{.T}}/env' + - echo '{{.T}}' + + trial:run: + desc: "Run one headless trial in a sandbox and check its isolation: task trial:run NAME= PROMPT='…' [DIR=] [SEEDED=true for an upgrade trial]." + requires: + vars: [NAME, PROMPT] + vars: + T: '{{.TRIALS}}/{{.NAME}}' + DIR: '{{.DIR | default ""}}' + SEEDED: '{{if eq (.SEEDED | default "") "true"}}--seeded{{end}}' + VERSION: + sh: sed -n 's/^version = "\(.*\)"$/\1/p' Cargo.toml | head -1 + cmds: + - cp '{{.T}}/home/.claude/plugins/installed_plugins.json' '{{.T}}/registry-at-start.json' + - install -m 600 "$HOME/.claude/.credentials.json" '{{.T}}/home/.claude/.credentials.json' + - defer: rm -f '{{.T}}/home/.claude/.credentials.json' + - | + (. '{{.T}}/env' && mkdir -p '{{.T}}/work/{{.DIR}}' && cd '{{.T}}/work/{{.DIR}}' && + timeout 2400 claude -p "$PROMPT" --permission-mode acceptEdits \ + --allowedTools 'Bash(b10x:*)' 'Bash(ess:*)' 'Bash(aep:*)' 'Bash(worktree:*)' 'Bash(git:*)' \ + 'Bash(mkdir:*)' 'Bash(ls:*)' 'Bash(cat:*)' 'Bash(find:*)' 'Bash(curl:*)' 'Bash(tar:*)' \ + 'Bash(sha256sum:*)' 'Bash(mv:*)' 'Bash(chmod:*)' Read Write Edit Glob Grep Skill Task Agent \ + --strict-mcp-config --output-format stream-json --verbose > '{{.T}}/run.jsonl' 2> '{{.T}}/run.err' + echo "EXIT=$?" >> '{{.T}}/run.err') + - cargo run --quiet --locked --bin agentplugins-check -- trial-isolation '{{.T}}/run.jsonl' --sandbox '{{.T}}' --marketplace '{{.ROOT_DIR}}' --version '{{.VERSION}}' {{.SEEDED}} + env: + PROMPT: '{{.PROMPT}}' + site-build: desc: Install and build the public Docusaurus documentation (network). cmds: diff --git a/crates/agentplugins-check/Cargo.toml b/crates/agentplugins-check/Cargo.toml index ad90343..f7acde5 100644 --- a/crates/agentplugins-check/Cargo.toml +++ b/crates/agentplugins-check/Cargo.toml @@ -9,6 +9,7 @@ repository.workspace = true authors.workspace = true [dependencies] +clap.workspace = true serde.workspace = true serde_json.workspace = true serde_yaml.workspace = true diff --git a/crates/agentplugins-check/src/concept.rs b/crates/agentplugins-check/src/concept.rs index 685015a..bbcbd54 100644 --- a/crates/agentplugins-check/src/concept.rs +++ b/crates/agentplugins-check/src/concept.rs @@ -6,7 +6,8 @@ //! - **R3** a skill is an activity named in `-ing` form, one or two words, never its plugin's name. //! - **R4** an agent is owned by exactly one skill of its plugin, which lists it under `## Agents`. //! - **R7** every `:` id written in this repository resolves. -//! - **R8** one README row, one plugin page and one sidebar entry per plugin; the README stays short. +//! - **R8** one README row, one plugin page and one sidebar entry per plugin; the README stays short, +//! besides a generated tree of every skill and agent. //! //! R1, R5 and R6 are the marketplace, manifest-version and retired-name checks in `main.rs`. @@ -390,21 +391,86 @@ pub fn readme_rows(readme: &str) -> Vec { .collect() } +/// Markers around the README's plugin tree, which [`tree`] generates. +const TREE_START: &str = ""; +const TREE_END: &str = ""; + +/// One carried plugin's skills and agents, by name. +pub struct Contents { + /// Skill folder names. + pub skills: BTreeSet, + /// Agent file stems. + pub agents: BTreeSet, +} + +/// The README's plugin tree, in table order: every skill (lifecycle first) and agent, each linked +/// to its source file. +#[must_use] +pub fn tree(order: &[String], contents: &BTreeMap) -> String { + let mut lines = vec![TREE_START.to_owned()]; + for name in order { + let Some(plugin) = contents.get(name) else { + continue; + }; + lines.push(format!( + "- [`{name}`](plugins/{name}/) · [docs](website/docs/plugins/{name}.md)" + )); + let lifecycle = LIFECYCLE.iter().map(|s| (*s).to_owned()); + let rest = plugin + .skills + .iter() + .filter(|s| !LIFECYCLE.contains(&s.as_str())) + .cloned(); + let skills: Vec = lifecycle + .chain(rest) + .filter(|s| plugin.skills.contains(s)) + .map(|s| format!("[`{s}`](plugins/{name}/skills/{s}/SKILL.md)")) + .collect(); + lines.push(format!(" - skills: {}", skills.join(" · "))); + if !plugin.agents.is_empty() { + let agents: Vec = plugin + .agents + .iter() + .map(|a| format!("[`{a}`](plugins/{name}/agents/{a}.md)")) + .collect(); + lines.push(format!(" - agents: {}", agents.join(" · "))); + } + } + lines.push(TREE_END.to_owned()); + lines.join("\n") +} + /// R8. -fn docs(root: &Path, plugins: &Plugins, problems: &mut Vec) -> Result<(), String> { +fn docs( + root: &Path, + plugins: &Plugins, + contents: &BTreeMap, + problems: &mut Vec, +) -> Result<(), String> { let all = plugins.all(); let readme = read(&root.join("README.md"))?; let head = readme .split("") .next() .unwrap_or_default(); - if head.lines().count() > README_LINES { + let written = match (head.find(TREE_START), head.find(TREE_END)) { + (Some(start), Some(end)) if start < end => &head[start..end + TREE_END.len()], + _ => "", + }; + let prose = head.lines().count() - written.lines().count(); + if prose > README_LINES { problems.push(format!( - "R8 README.md has {} lines before its documentation block; the most is {README_LINES}: one paragraph, the plugin table, one link line", - head.lines().count() + "R8 README.md has {prose} lines before its documentation block, besides the plugin tree; the most is {README_LINES}: one paragraph, the plugin table, one link line", )); } - let rows: BTreeSet = readme_rows(head).into_iter().collect(); + let rows = readme_rows(head); + let expected = tree(&rows, contents); + if written != expected { + problems.push(format!( + "R8 README.md plugin tree is missing or stale; it must read:\n{expected}" + )); + } + let rows: BTreeSet = rows.into_iter().collect(); if rows != all { problems.push(format!( "R8 README.md table lists {rows:?}; the catalog has {all:?}" @@ -441,15 +507,17 @@ pub fn check(root: &Path, plugins: &Plugins) -> Result<(), String> { .map_err(|error| format!("parsing catalog.json: {error}"))?; products(&catalog, &mut problems); let mut known = BTreeMap::new(); + let mut contents = BTreeMap::new(); for name in &plugins.carried { let (skills, agents) = plugin(root, name, &mut problems); known.insert( name.clone(), skills.union(&agents).cloned().collect::>(), ); + contents.insert(name.clone(), Contents { skills, agents }); } references(root, plugins, &known, &mut problems); - docs(root, plugins, &mut problems)?; + docs(root, plugins, &contents, &mut problems)?; if problems.is_empty() { Ok(()) } else { @@ -521,6 +589,36 @@ mod tests { ); } + #[test] + fn the_tree_lists_lifecycle_skills_first_and_links_every_file() { + let mut contents = BTreeMap::new(); + contents.insert( + "ess".to_owned(), + Contents { + skills: ["specifying", "upgrade", "init"].map(str::to_owned).into(), + agents: ["author"].map(str::to_owned).into(), + }, + ); + contents.insert( + "b10x".to_owned(), + Contents { + skills: ["init", "upgrade"].map(str::to_owned).into(), + agents: BTreeSet::new(), + }, + ); + let order = ["ess".to_owned(), "b10x".to_owned()]; + assert_eq!( + tree(&order, &contents), + "\n\ + - [`ess`](plugins/ess/) · [docs](website/docs/plugins/ess.md)\n \ + - skills: [`init`](plugins/ess/skills/init/SKILL.md) · [`upgrade`](plugins/ess/skills/upgrade/SKILL.md) · [`specifying`](plugins/ess/skills/specifying/SKILL.md)\n \ + - agents: [`author`](plugins/ess/agents/author.md)\n\ + - [`b10x`](plugins/b10x/) · [docs](website/docs/plugins/b10x.md)\n \ + - skills: [`init`](plugins/b10x/skills/init/SKILL.md) · [`upgrade`](plugins/b10x/skills/upgrade/SKILL.md)\n\ + " + ); + } + #[test] fn readme_rows_read_the_first_column() { let readme = "| plugin | for |\n|---|---|\n| [`ess`](x) | a |\n| [`aep`](y) | b |\n"; diff --git a/crates/agentplugins-check/src/evals.rs b/crates/agentplugins-check/src/evals.rs index 4d14dc5..abd2866 100644 --- a/crates/agentplugins-check/src/evals.rs +++ b/crates/agentplugins-check/src/evals.rs @@ -936,39 +936,44 @@ fn skill_directory(root: &Path, plugin: &str, name: &str, case: &str) -> Result< )) } -/// The corpus subcommands, or [`None`] when these arguments are not one. +/// The corpus subcommands. +#[derive(Debug, clap::Subcommand)] +pub(crate) enum Action { + /// Check one live run: its command contract and the AEP report beside the stream. + CheckStream { + /// The case directory, relative to the repository root. + case: std::path::PathBuf, + /// The `.events.jsonl` stream the run wrote. + stream: std::path::PathBuf, + }, + /// Print the cases (and their plugins) that the changed paths concern. + Scope { + /// Changed repository-relative paths. + changed: Vec, + }, +} + +/// Run a corpus subcommand; none validates the corpus and replays every recorded transcript. /// -/// Handled before the marketplace check's own argument match and never inside it, so that adding -/// this surface moved three lines of `main` and touched no existing arm. The `Ok` arm there prints -/// *marketplace b10x* and would be a false sentence under either verb below. -pub(crate) fn cli(root: &Path, arguments: &[String]) -> Option { - let verdict = match arguments - .iter() - .map(String::as_str) - .collect::>() - .as_slice() - { - ["evals"] => evals(root), - ["evals", "check-stream", case, stream] => { - check_stream(root, Path::new(case), Path::new(stream)) - } - ["evals", "scope", changed @ ..] => { - let changed: Vec = changed.iter().map(|path| (*path).to_owned()).collect(); - scope(root, &changed).map(|matched| { - for (case, plugin) in matched { - println!("{case}\t{plugin}"); - } - }) - } - _ => return None, +/// Kept apart from the marketplace check, whose success line prints *marketplace b10x* and would be +/// a false sentence under either verb here. +pub(crate) fn run(root: &Path, action: Option) -> std::process::ExitCode { + let verdict = match action { + None => evals(root), + Some(Action::CheckStream { case, stream }) => check_stream(root, &case, &stream), + Some(Action::Scope { changed }) => scope(root, &changed).map(|matched| { + for (case, plugin) in matched { + println!("{case}\t{plugin}"); + } + }), }; - Some(match verdict { + match verdict { Ok(()) => std::process::ExitCode::SUCCESS, Err(error) => { eprintln!("error: {error}"); std::process::ExitCode::from(1) } - }) + } } fn check_commands(case: &Case, stream: &Path) -> Result<(), String> { diff --git a/crates/agentplugins-check/src/main.rs b/crates/agentplugins-check/src/main.rs index ba9f7c6..5d3fa62 100644 --- a/crates/agentplugins-check/src/main.rs +++ b/crates/agentplugins-check/src/main.rs @@ -3,10 +3,13 @@ use std::path::{Path, PathBuf}; use std::process::{Command, ExitCode}; +use clap::{Parser, Subcommand}; + mod concept; mod evals; mod readiness; mod tools; +mod trial; /// The marketplace identity in every marketplace format. const MARKETPLACE: &str = "b10x"; @@ -335,13 +338,12 @@ const RETIRED: &[Retired] = &[ new: "aep@b10x", wire_next: &[], }, - // ESS ships its own plugin from its repository; this marketplace points at it. + // ESS and worktree plugins live here under `b10x` (website/docs/structure.md R1). Retired { old: "ess-specify", new: "ess@b10x", wire_next: &[], }, - // Worktree ships its own plugin from its repository; this marketplace points at it. Retired { old: "workspace-hygiene", new: "worktree@b10x", @@ -411,7 +413,7 @@ const RETIRED: &[Retired] = &[ }, Retired { old: "b10x:setup", - new: "b10x:installing", + new: "b10x:init", wire_next: &[], }, Retired { @@ -429,6 +431,17 @@ const RETIRED: &[Retired] = &[ new: "connectors:integrating", wire_next: &[], }, + // The `ess` binary no longer prints skills; `b10x skill` does. + Retired { + old: "ess skill", + new: "b10x skill ess:", + wire_next: &[], + }, + Retired { + old: "beyond10x/ess` marketplace", + new: "the b10x marketplace", + wire_next: &[], + }, ]; /// Directories never walked looking for a retired name, wherever they sit. @@ -945,26 +958,88 @@ fn verify_release(root: &Path, version: &str) -> Result<(), String> { Ok(()) } +/// Validates the Beyond10x agent plugin marketplace. With no subcommand: the offline gate. +#[derive(Debug, Parser)] +#[command(name = "agentplugins-check")] +struct Cli { + #[command(subcommand)] + command: Option, +} + +#[derive(Debug, Subcommand)] +enum Top { + /// The offline gate, then every spelled CLI command against the newest releases (network). + Tools, + /// Release checks. + Release { + #[command(subcommand)] + action: ReleaseAction, + }, + /// Validate the eval corpus and replay recorded transcripts, or one of the corpus verbs. + Evals { + #[command(subcommand)] + action: Option, + }, + /// Check that a headless trial run loaded only the sandbox's plugins, at the version under test. + TrialIsolation { + /// The run's stream-json output. + run: PathBuf, + /// The sandbox directory (its `home/` is the run's `HOME`). + #[arg(long)] + sandbox: PathBuf, + /// A directory marketplace the plugins may load from in place (the checkout under test). + #[arg(long)] + marketplace: Option, + /// The agentplugins version under test. + #[arg(long)] + version: String, + /// An upgrade trial: plugins start at the versions the sandbox was seeded with. + #[arg(long)] + seeded: bool, + }, +} + +#[derive(Debug, Subcommand)] +enum ReleaseAction { + /// Check that every version in the repository agrees with the tag. + Verify { + /// The tag being released. + version: String, + }, +} + fn main() -> ExitCode { let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")) .parent() .and_then(Path::parent) .expect("checker is under the repository root") .to_path_buf(); - let arguments = std::env::args().skip(1).collect::>(); - if let Some(code) = evals::cli(&root, &arguments) { - return code; - } - let result = match arguments.as_slice() { - [] => check(&root), - [tools] if tools == "tools" => check(&root).and_then(|()| tools::verify(&root)), - [release, verify, version] if release == "release" && verify == "verify" => { - check(&root).and_then(|()| verify_release(&root, version)) + let result = match Cli::parse().command { + None => check(&root), + Some(Top::Tools) => check(&root).and_then(|()| tools::verify(&root)), + Some(Top::Release { + action: ReleaseAction::Verify { version }, + }) => check(&root).and_then(|()| verify_release(&root, &version)), + Some(Top::Evals { action }) => return evals::run(&root, action), + Some(Top::TrialIsolation { + run, + sandbox, + marketplace, + version, + seeded, + }) => { + return match trial::isolation(&run, &sandbox, marketplace.as_deref(), &version, seeded) + { + Ok(summary) => { + println!("{summary}"); + ExitCode::SUCCESS + } + Err(error) => { + eprintln!("error: {error}"); + ExitCode::from(1) + } + }; } - _ => Err( - "usage: agentplugins-check [tools | release verify | evals | evals scope ...]" - .to_owned(), - ), }; match result { diff --git a/crates/agentplugins-check/src/tools.rs b/crates/agentplugins-check/src/tools.rs index 9ecd42f..f01a5db 100644 --- a/crates/agentplugins-check/src/tools.rs +++ b/crates/agentplugins-check/src/tools.rs @@ -215,6 +215,46 @@ fn syntax(root: &Path, ess: &Path, scratch: &Path) -> Result<(), String> { ) })?; println!("tools `ess`: syntax example — {}", out.trim()); + let ess = ess.to_string_lossy(); + let path = dir.to_string_lossy(); + let suite = scratch.join("syntax-suite.json"); + let generated = scratch.join("syntax-generated"); + let generated = generated.to_string_lossy(); + let synthesized = run( + &ess, + &[ + "verify", + "conform", + "synthesize", + "--path", + &path, + "--out", + &suite.to_string_lossy(), + ], + ) + .map_err(|error| format!("{}: synthesize failed: {error}", reference.display()))?; + if !synthesized.contains(" 0 refusal(s)") { + return Err(format!( + "{}: the example synthesizes with refusals: {}", + reference.display(), + synthesized.trim() + )); + } + println!("tools `ess`: syntax example — {}", synthesized.trim()); + for arguments in [ + &["generate", "--kind", "schema"][..], + &["generate", "project", "openapi"][..], + ] { + let mut arguments = arguments.to_vec(); + arguments.extend(["--path", &path, "--out", &generated]); + run(&ess, &arguments).map_err(|error| { + format!( + "{}: `ess {}` failed on the example: {error}", + reference.display(), + arguments[..arguments.len() - 4].join(" ") + ) + })?; + } Ok(()) } diff --git a/crates/agentplugins-check/src/trial.rs b/crates/agentplugins-check/src/trial.rs new file mode 100644 index 0000000..c729093 --- /dev/null +++ b/crates/agentplugins-check/src/trial.rs @@ -0,0 +1,290 @@ +//! Whether a headless trial run tested what it claims to: the plugins in its sandbox, at the +//! version under test, and nothing from the machine it ran on. +//! +//! A trial run as a sub-agent of a working session inherits that session's agents, so its critics +//! were the host's and not the plugin's under test (trial 3, 2026-09-25). The run's own `init` +//! event lists what it loaded; this reads it and refuses anything that did not come from the +//! sandbox. +//! +//! A plugin the sandbox never installed is refused too, wherever it loaded from: Claude Code also +//! reads `.claude/settings*.json` in the directories above the working directory, and one left in +//! `~/.cache` by an earlier trial enabled `aep@b10x` in every sandbox below it (trial 3). + +use std::collections::{BTreeMap, BTreeSet}; +use std::path::{Path, PathBuf}; + +use serde_json::Value; + +/// The marketplace every plugin under test comes from. +const MARKETPLACE: &str = "b10x"; + +/// What the sandbox itself provides. +struct Sandbox<'a> { + /// The sandbox directory; its `home/` is the run's `HOME`. + root: &'a Path, + /// A directory marketplace plugins may load from in place (the checkout under test). + marketplace: Option<&'a Path>, + /// `name@marketplace` of every plugin the sandbox's own registry held when the run started, + /// with the versions it held. + installed: BTreeMap>, + /// The sandbox was seeded with older plugins on purpose (an upgrade trial): a plugin must be at + /// the version the sandbox installed, not at the version under test. + seeded: bool, +} + +/// Check every `init` event of a stream-json run; returns a one-line summary. +pub fn isolation( + run: &Path, + sandbox: &Path, + marketplace: Option<&Path>, + version: &str, + seeded: bool, +) -> Result { + let text = std::fs::read_to_string(run) + .map_err(|error| format!("reading {}: {error}", run.display()))?; + let root = std::fs::canonicalize(sandbox) + .map_err(|error| format!("sandbox {}: {error}", sandbox.display()))?; + let marketplace = marketplace + .map(std::fs::canonicalize) + .transpose() + .map_err(|error| format!("marketplace: {error}"))?; + // `task trial:run` copies the registry before the run: an upgrade changes it during the run. + let registry = [ + root.join("registry-at-start.json"), + root.join("home/.claude/plugins/installed_plugins.json"), + ] + .into_iter() + .find(|path| path.is_file()) + .ok_or_else(|| format!("{} has no plugin registry", root.display()))?; + let registry: Value = std::fs::read_to_string(®istry) + .ok() + .and_then(|text| serde_json::from_str(&text).ok()) + .ok_or_else(|| format!("reading {}", registry.display()))?; + let installed = registry["plugins"] + .as_object() + .map(|plugins| { + plugins + .iter() + .map(|(id, entries)| { + let versions = entries + .as_array() + .into_iter() + .flatten() + .filter_map(|entry| entry["version"].as_str().map(str::to_owned)) + .collect(); + (id.clone(), versions) + }) + .collect() + }) + .unwrap_or_default(); + check( + &text, + &Sandbox { + root: &root, + marketplace: marketplace.as_deref(), + installed, + seeded, + }, + version, + ) +} + +#[allow(clippy::too_many_lines)] +fn check(text: &str, sandbox: &Sandbox<'_>, version: &str) -> Result { + let root = sandbox.root; + let mut problems = Vec::new(); + let mut inits = 0; + let mut tested = BTreeSet::new(); + for (number, line) in text.lines().enumerate() { + let Ok(event) = serde_json::from_str::(line) else { + continue; + }; + if event["type"] != "system" || event["subtype"] != "init" { + continue; + } + inits += 1; + let at = format!("line {}", number + 1); + if let Some(cwd) = event["cwd"].as_str() { + if !Path::new(cwd).starts_with(root) { + problems.push(format!("{at}: ran in {cwd}, outside the sandbox")); + } + } + for server in event["mcp_servers"].as_array().into_iter().flatten() { + let name = server["name"].as_str().unwrap_or("?"); + problems.push(format!( + "{at}: MCP server `{name}` was connected; trials run with `--strict-mcp-config` and none" + )); + } + let mut loaded = BTreeSet::new(); + for plugin in event["plugins"].as_array().into_iter().flatten() { + let name = plugin["name"].as_str().unwrap_or_default(); + let path = plugin["path"].as_str().unwrap_or_default(); + let source = plugin["source"].as_str().unwrap_or_default(); + loaded.insert(name.to_owned()); + if path == "builtin" { + continue; + } + let path = PathBuf::from(path); + let in_place = sandbox + .marketplace + .is_some_and(|marketplace| path.starts_with(marketplace)); + if !path.starts_with(root.join("home")) && !in_place { + problems.push(format!( + "{at}: plugin `{source}` loaded from {}, outside the sandbox home and the marketplace under test", + path.display() + )); + } + if !sandbox.installed.contains_key(source) { + problems.push(format!( + "{at}: plugin `{source}` is not in the sandbox's installed_plugins.json; a settings file outside the sandbox enabled it" + )); + } + if source.ends_with(&format!("@{MARKETPLACE}")) { + tested.insert(name.to_owned()); + let loaded_version = plugin["version"].as_str().unwrap_or_default(); + let expected = if sandbox.seeded { + sandbox + .installed + .get(source) + .is_some_and(|versions| versions.contains(loaded_version)) + } else { + loaded_version == version + }; + if !expected { + problems.push(format!( + "{at}: plugin `{source}` is {loaded_version}, not {}", + if sandbox.seeded { + "a version the sandbox was seeded with".to_owned() + } else { + format!("the version under test {version}") + } + )); + } + } else { + problems.push(format!( + "{at}: plugin `{source}` is not from the `{MARKETPLACE}` marketplace" + )); + } + } + for list in ["agents", "skills"] { + for id in event[list].as_array().into_iter().flatten() { + let id = id.as_str().unwrap_or_default(); + if let Some((owner, _)) = id.split_once(':') { + if !loaded.contains(owner) { + problems.push(format!( + "{at}: {} `{id}` belongs to no plugin the sandbox loaded", + list.trim_end_matches('s') + )); + } + } + } + } + } + if inits == 0 { + problems.push( + "no `init` event: not a stream-json run (`--output-format stream-json --verbose`)" + .to_owned(), + ); + } else if tested.is_empty() { + problems.push(format!("no `@{MARKETPLACE}` plugin was loaded")); + } + if problems.is_empty() { + let tested: Vec = tested.into_iter().collect(); + Ok(format!( + "isolated: {inits} session(s), plugins {} at {version}, all from {}", + tested.join(", "), + root.display() + )) + } else { + Err(format!( + "the run is not an isolated trial of {version}:\n {}", + problems.join("\n ") + )) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn init(plugins: &str, agents: &str) -> String { + format!( + r#"{{"type":"system","subtype":"init","cwd":"/opt/trial/work","plugins":[{plugins}],"agents":[{agents}],"skills":["ess:specifying","verify"]}}"# + ) + } + + fn sandbox() -> Sandbox<'static> { + Sandbox { + root: Path::new("/opt/trial"), + marketplace: Some(Path::new("/opt/checkout")), + installed: [("ess@b10x".to_owned(), ["0.14.3".to_owned()].into())].into(), + seeded: false, + } + } + + #[test] + fn a_seeded_upgrade_trial_starts_from_the_seeded_version() { + let old = r#"{"name":"ess","path":"/opt/trial/home/.claude/plugins/cache/b10x/ess/0.12.0","source":"ess@b10x","version":"0.12.0"}"#; + let mut seeded = sandbox(); + seeded.installed = [("ess@b10x".to_owned(), ["0.12.0".to_owned()].into())].into(); + seeded.seeded = true; + check(&init(old, ""), &seeded, "0.14.3").unwrap(); + seeded.seeded = false; + let error = check(&init(old, ""), &seeded, "0.14.3").unwrap_err(); + assert!(error.contains("not the version under test"), "{error}"); + } + + #[test] + fn a_plugin_the_sandbox_never_installed_fails_the_run() { + let leaked = r#"{"name":"aep","path":"/opt/checkout/plugins/aep","source":"aep@b10x","version":"0.14.3"}"#; + let error = check(&init(leaked, ""), &sandbox(), "0.14.3").unwrap_err(); + assert!( + error.contains("not in the sandbox's installed_plugins.json"), + "{error}" + ); + assert!(!error.contains("outside the sandbox home"), "{error}"); + } + + const SANDBOXED: &str = r#"{"name":"ess","path":"/opt/trial/home/.claude/plugins/cache/b10x/ess/0.14.3","source":"ess@b10x","version":"0.14.3"},{"name":"telemetry","path":"builtin","source":"telemetry@builtin"}"#; + + #[test] + fn a_sandboxed_run_passes() { + let run = init(SANDBOXED, r#""claude","ess:author""#); + let summary = check(&run, &sandbox(), "0.14.3").unwrap(); + assert!(summary.contains("plugins ess at 0.14.3"), "{summary}"); + } + + #[test] + fn a_host_agent_fails_the_run() { + let run = init(SANDBOXED, r#""host-plugin:plan-critic-design""#); + let error = check(&run, &sandbox(), "0.14.3").unwrap_err(); + assert!( + error.contains("agent `host-plugin:plan-critic-design`"), + "{error}" + ); + } + + #[test] + fn another_version_or_a_host_path_fails_the_run() { + let host = r#"{"name":"ess","path":"/opt/host/.claude/plugins/cache/b10x/ess/0.14.2","source":"ess@b10x","version":"0.14.2"}"#; + let error = check(&init(host, ""), &sandbox(), "0.14.3").unwrap_err(); + assert!(error.contains("outside the sandbox home"), "{error}"); + assert!(error.contains("not the version under test"), "{error}"); + } + + #[test] + fn an_account_mcp_server_fails_the_run() { + let run = init(SANDBOXED, "").replace( + "\"agents\"", + "\"mcp_servers\":[{\"name\":\"claude.ai Docs\",\"status\":\"connected\"}],\"agents\"", + ); + let error = check(&run, &sandbox(), "0.14.3").unwrap_err(); + assert!(error.contains("MCP server `claude.ai Docs`"), "{error}"); + } + + #[test] + fn a_stream_without_init_fails() { + let error = check("{\"type\":\"result\"}", &sandbox(), "0.14.3").unwrap_err(); + assert!(error.contains("no `init` event"), "{error}"); + } +} diff --git a/crates/b10x/src/check.rs b/crates/b10x/src/check.rs index a8ee71c..eab262e 100644 --- a/crates/b10x/src/check.rs +++ b/crates/b10x/src/check.rs @@ -128,7 +128,7 @@ pub fn lines( } } else if !installed .iter() - .any(|plugin| catalog.product_of(plugin).is_some()) + .any(|plugin| catalog.product_of(catalog.current_name(plugin)).is_some()) { lines.push( "b10x: no Beyond10x product is set up yet; /b10x:init asks what you want to do and installs it." @@ -230,4 +230,15 @@ mod tests { assert_eq!(out.len(), 1); assert!(out[0].contains("/b10x:init")); } + + #[test] + fn a_legacy_plugin_is_not_nothing_set_up() { + let catalog = Catalog::embedded(); + let plugins = [ + plugin("b10x", "b10x", "0.12.0"), + plugin("aep-plan", "b10x", "0.12.0"), + ]; + let out = lines(&catalog, &plugins, &|_: &str| None, None, 0); + assert!(!out.iter().any(|l| l.contains("/b10x:init")), "{out:#?}"); + } } diff --git a/crates/b10x/src/main.rs b/crates/b10x/src/main.rs index 6422053..a7f815f 100644 --- a/crates/b10x/src/main.rs +++ b/crates/b10x/src/main.rs @@ -132,7 +132,7 @@ enum Setup { /// Snapshot directory or its timestamp name. snapshot: Option, }, - /// Print the setup instructions an agent follows (the `b10x:installing` skill). + /// Print the setup instructions an agent follows (the `b10x:init` skill). Guide, } @@ -292,7 +292,7 @@ fn make_plan( if let Some(source) = &overridden { embedded.marketplace.repository.clone_from(source); } - let inventory = inventory::collect(&embedded, &hosts); + let inventory = inventory::collect(&embedded, &[Host::Claude, Host::Codex]); let local = overridden .as_deref() .map(std::path::Path::new) @@ -457,7 +457,7 @@ fn setup_apply(path: &PathBuf, yes: bool) -> Result { )); } let embedded = Catalog::embedded(); - let inventory = inventory::collect(&embedded, &plan.hosts); + let inventory = inventory::collect(&embedded, &[Host::Claude, Host::Codex]); apply::fresh(&plan, &inventory)?; if !yes { print_plan(&plan); diff --git a/crates/b10x/src/plan.rs b/crates/b10x/src/plan.rs index b97e12e..4718601 100644 --- a/crates/b10x/src/plan.rs +++ b/crates/b10x/src/plan.rs @@ -295,13 +295,7 @@ pub fn plan(context: &Context<'_>, inventory: &Inventory) -> Plan { Host::Claude => inventory.claude.as_ref(), Host::Codex => inventory.codex.as_ref(), }; - let untouched = state.is_some_and(|state| { - !state.plugins.iter().any(|plugin| { - plugin.marketplace == catalog.marketplace.name - || catalog.retired_marketplace(&plugin.marketplace) - || catalog.retired_plugins.contains_key(&plugin.name) - }) - }); + let untouched = state.is_some_and(|state| !holds_beyond10x(catalog, state)); if context.upgrade && untouched { findings.push(Finding { level: Level::Note, @@ -345,6 +339,24 @@ pub fn plan(context: &Context<'_>, inventory: &Inventory) -> Plan { }, } } + for (host, state) in [ + (Host::Claude, inventory.claude.as_ref()), + (Host::Codex, inventory.codex.as_ref()), + ] { + if state.is_some_and(|state| holds_beyond10x(catalog, state)) + && !context.hosts.contains(&host) + { + findings.push(Finding { + level: Level::Note, + host: Some(host), + subject: host.program().to_owned(), + detail: format!( + "Beyond10x plugins are installed for `{}` too and this plan leaves them alone; `--host all` covers both", + host.program() + ), + }); + } + } if context.hosts.contains(&Host::Claude) && inventory.claude.is_some() { plan_settings(context, &selected, inventory, &mut findings, &mut actions); } @@ -375,6 +387,15 @@ pub fn plan(context: &Context<'_>, inventory: &Inventory) -> Plan { } } +/// Whether a host holds anything from Beyond10x: a current plugin, or a legacy one. +fn holds_beyond10x(catalog: &Catalog, state: &HostState) -> bool { + state.plugins.iter().any(|plugin| { + plugin.marketplace == catalog.marketplace.name + || catalog.retired_marketplace(&plugin.marketplace) + || catalog.retired_plugins.contains_key(&plugin.name) + }) +} + fn scope_args(plugin: &Installed) -> Vec { argv(&["--scope", &plugin.scope]) } @@ -478,11 +499,25 @@ fn plan_host( .filter(|plugin| plugin.marketplace == name && plugin.scope == "user") .map(|plugin| (plugin.name.as_str(), plugin)) .collect(); + // Plugins a legacy install on this host stands in for: section 3 removes the legacy install, + // so an upgrade installs its replacement instead of only noting that it is missing. + let replacing: BTreeSet<&str> = state + .plugins + .iter() + .filter(|plugin| { + catalog.retired_marketplace(&plugin.marketplace) + || catalog.retired_plugins.contains_key(&plugin.name) + }) + .map(|plugin| catalog.current_name(&plugin.name)) + .collect(); for plugin in &desired { let id = format!("{plugin}@{name}"); let target = context.resolved.plugins.get(*plugin); match user.get(plugin) { - None if context.upgrade && !catalog.base.iter().any(|base| base == plugin) => { + None if context.upgrade + && !catalog.base.iter().any(|base| base == plugin) + && !replacing.contains(plugin) => + { findings.push(finding( Level::Note, &id, @@ -575,7 +610,10 @@ fn plan_host( plugin.scope ), )); - if plugin.scope != "user" { + // A local install is this user's alone, so the user-scope install of a selected product + // already covers it. A project install is committed for everyone and is replaced in place. + let covered = plugin.scope == "local" && desired.contains(&replacement.as_str()); + if plugin.scope != "user" && !covered { let already = state.plugins.iter().any(|other| { other.name == replacement && other.marketplace == name @@ -763,7 +801,13 @@ fn plan_settings( && installed.scope == entry.scope && installed.project == entry.project }); - if entry.enabled && entry.scope != "user" && !already && catalog.knows(replacement) { + let covered = entry.scope == "local" && wanted(catalog, selected, replacement); + if entry.enabled + && entry.scope != "user" + && !covered + && !already + && catalog.knows(replacement) + { actions.push(install( Host::Claude, &target, @@ -1201,7 +1245,7 @@ mod tests { } #[test] - fn an_orphan_in_a_project_file_is_removed_and_replaced_in_place() { + fn a_local_orphan_is_removed_and_its_replacement_goes_to_user_scope() { let mut state = HostState::default(); state.marketplaces.push(Market { name: "b10x".to_owned(), @@ -1224,9 +1268,122 @@ mod tests { assert!(plan.actions.iter().any( |a| matches!(a, Action::RemoveSetting { key, .. } if key == "beyond10x@beyond10x") )); - assert!(plan.actions.iter().any(|a| matches!(a, - Action::Command { argv, cwd: Some(cwd), .. } - if argv.join(" ") == "claude plugin install b10x@b10x --scope local" && cwd == "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/work/acd"))); + let commands = commands(&plan); + assert!( + !commands.iter().any(|c| c.contains("--scope local")), + "{commands:#?}" + ); + assert!(commands.contains(&"claude plugin install b10x@b10x --scope user".to_owned())); + } + + fn market() -> Market { + Market { + name: "b10x".to_owned(), + source: "beyond10x/agentplugins".to_owned(), + reference: None, + location: None, + } + } + + fn installed(name: &str, marketplace: &str, scope: &str, project: Option<&str>) -> Installed { + Installed { + name: name.to_owned(), + marketplace: marketplace.to_owned(), + version: Some("0.12.0".to_owned()), + scope: scope.to_owned(), + project: project.map(str::to_owned), + enabled: true, + } + } + + fn run_upgrade(inventory: &Inventory, hosts: &[Host]) -> Plan { + let catalog = Catalog::embedded(); + let resolved = resolved(); + let context = Context { + catalog: &catalog, + resolved: &resolved, + selection: None, + hosts: hosts.to_vec(), + home: Path::new("/opt/b10x-home"), + only: true, + method: None, + upgrade: true, + }; + plan(&context, inventory) + } + + #[test] + fn upgrade_installs_the_replacement_of_a_legacy_plugin_it_removes() { + let mut claude = HostState::default(); + claude.marketplaces.push(market()); + claude.plugins.push(installed("b10x", "b10x", "user", None)); + claude + .plugins + .push(installed("worktree", "worktree", "user", None)); + let inventory = Inventory { + claude: Some(claude), + ..Inventory::default() + }; + let commands = commands(&run_upgrade(&inventory, &[Host::Claude])); + assert!( + commands.contains(&"claude plugin uninstall worktree@worktree --scope user".to_owned()), + "{commands:#?}" + ); + assert!( + commands.contains(&"claude plugin install worktree@b10x --scope user".to_owned()), + "{commands:#?}" + ); + } + + #[test] + fn a_legacy_local_install_the_user_scope_covers_is_only_removed() { + let mut claude = HostState::default(); + claude.marketplaces.push(market()); + claude + .plugins + .push(installed("aep-plan", "b10x", "local", Some("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/work/p"))); + claude + .plugins + .push(installed("aep-drive", "b10x", "project", Some("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/work/q"))); + let inventory = Inventory { + claude: Some(claude), + ..Inventory::default() + }; + let commands = commands(&run(&inventory, Some(&["aep"]), &[Host::Claude])); + assert!( + commands.contains(&"claude plugin install aep@b10x --scope user".to_owned()), + "{commands:#?}" + ); + assert!( + !commands + .iter() + .any(|c| c == "claude plugin install aep@b10x --scope local"), + "{commands:#?}" + ); + assert!( + commands.contains(&"claude plugin install aep@b10x --scope project".to_owned()), + "a committed project install is replaced in place: {commands:#?}" + ); + } + + #[test] + fn beyond10x_on_a_host_outside_the_plan_is_named() { + let mut codex = HostState::default(); + codex.plugins.push(installed("ess", "b10x", "user", None)); + let inventory = Inventory { + claude: Some(HostState::default()), + codex: Some(codex), + ..Inventory::default() + }; + let plan = run(&inventory, Some(&["ess"]), &[Host::Claude]); + assert!( + plan.findings + .iter() + .any(|f| f.host == Some(Host::Codex) && f.detail.contains("--host all")), + "{:#?}", + plan.findings + ); + assert!(!commands(&plan).iter().any(|c| c.starts_with("codex"))); } #[test] diff --git a/plugins/aep/.claude-plugin/plugin.json b/plugins/aep/.claude-plugin/plugin.json index 866fe8a..55a221f 100644 --- a/plugins/aep/.claude-plugin/plugin.json +++ b/plugins/aep/.claude-plugin/plugin.json @@ -2,7 +2,7 @@ "name": "aep", "displayName": "AEP", "description": "Plan governed work in the AEP artifact store and deliver it in reviewed waves: decomposition, plan critique, reverse engineering, story scoping, implementation and adversarial review.", - "version": "0.14.2", + "version": "0.14.3", "author": { "name": "Beyond10x" }, diff --git a/plugins/aep/.codex-plugin/plugin.json b/plugins/aep/.codex-plugin/plugin.json index 0f0c625..a3bbd5c 100644 --- a/plugins/aep/.codex-plugin/plugin.json +++ b/plugins/aep/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "aep", - "version": "0.14.2", + "version": "0.14.3", "description": "Plan governed work in the AEP artifact store and deliver it in reviewed waves.", "author": { "name": "Beyond10x" diff --git a/plugins/aep/agents/plan-critic-design.md b/plugins/aep/agents/plan-critic-design.md index c286a11..c575968 100644 --- a/plugins/aep/agents/plan-critic-design.md +++ b/plugins/aep/agents/plan-critic-design.md @@ -55,6 +55,12 @@ that legitimately touch one file and do not say so is theirs. edge, not a rewrite, and your reason field should say which edge would say it — read the name from `aep plan artifact relations` rather than supplying one from memory. +**An ordering edge that records a shared file is not a serialising chain by itself.** When an edge +exists because two items edit one file (the parallel-safety critic asks for exactly that edge), the +remaining choice is between that order and splitting the shared surface so the items no longer +collide. Report it as that trade-off, naming both options and the file; do not ask for the edge to +be removed. A chain is your finding only when the edges have no such reason written beside them. + ## What is not yours to say * **The number of items.** Four or nine is the drafter's judgement unless the shape is broken. diff --git a/plugins/aep/agents/plan-critic-parallel-safety.md b/plugins/aep/agents/plan-critic-parallel-safety.md index 40841d4..d95fefb 100644 --- a/plugins/aep/agents/plan-critic-parallel-safety.md +++ b/plugins/aep/agents/plan-critic-parallel-safety.md @@ -63,7 +63,9 @@ surface is a weaker claim, and the drafter is entitled to see which kind they ar ## What is not yours to say * **The order the items should be worked in.** You report which pairs collide; sequencing is the - operator's. + operator's. Every collision finding names both remedies, without choosing: an ordering edge that + records the shared file as its reason, or splitting the surface so the two items no longer share + it. The design critic judges the same pair with the same two options. * **Whether a collision is acceptable.** Some are, deliberately. Name it and let a person decide. * **Anything about items outside the set you were given.** You cannot see them and must not guess. * **A collision on a file that does not exist yet.** Two items that would both *create* one file do diff --git a/plugins/aep/skills/init/SKILL.md b/plugins/aep/skills/init/SKILL.md index 0f1c2ac..2015680 100644 --- a/plugins/aep/skills/init/SKILL.md +++ b/plugins/aep/skills/init/SKILL.md @@ -18,6 +18,9 @@ It prints what it would do, including the install method: `cargo` when a Rust to both are possible, re-run with `--method cargo` or `--method prebuilt` to match, show the plan, and after they confirm run `b10x setup apply --plan ~/.local/state/b10x/plan.json --yes`. +Planning models new data with ESS (`aep:planning` rule 7). If `ess --version` does not answer, offer +it in the same step: `b10x init aep ess --out …` plans both. + No `b10x`? Follow https://github.com/beyond10x/agentplugins/releases/latest/download/SETUP.md first. ## 2. First step here diff --git a/plugins/aep/skills/migrating/SKILL.md b/plugins/aep/skills/migrating/SKILL.md index 1904c9f..53eefc1 100644 --- a/plugins/aep/skills/migrating/SKILL.md +++ b/plugins/aep/skills/migrating/SKILL.md @@ -95,6 +95,16 @@ things if it is not held to a rule. ### Step 4 — write +**The store must exist first.** Where `.engineering/project.yaml` is missing, create it with +`aep plan reverse init`; the `aep:planning` skill § 5 gives the `--protocols` and `--profile` +values. That applies with or without a backlog; only the steps after it differ. + +**A backlog item that introduces a new noun gets its domain first.** The `aep:planning` rule 7 +holds during a migration too: where an item names an entity no ESS document declares, draft and +validate the domain (`ess:specifying`) before writing the stories around it, and cite the file in +their bodies. Without `ess` installed, run `b10x init ess` and ask before applying its plan. The +classification table still lists the item; its stories are written after the domain validates. + One artifact at a time, body supplied at creation: ```console @@ -193,7 +203,7 @@ references — there is no door for an arbitrary key. So dates live in the body: Migrated from `.agents/plans/DEV-630_dispatch-retry-backoff.md`. -- First written 2026-06-16 · last touched 2026-06-16 · 1 revision +- Source first written 2026-06-16 · last touched 2026-06-16 · 1 git revision of the source - Status quoted from that file, line 17: **PLANNED — not yet implemented** - Ticket [DEV-630](https://example.atlassian.net/browse/DEV-630) ``` @@ -203,7 +213,7 @@ The three git facts, in order: ```console $ git log --follow --diff-filter=A --format=%aI -- | tail -1 # first written $ git log -1 --format=%cI -- # last touched -$ git log --oneline --follow -- | wc -l # revisions +$ git log --oneline --follow -- | wc -l # git revisions of the source ``` Quote dates the document states about itself too — `**Option A … LANDED** (2026-04-28)`, diff --git a/plugins/aep/skills/planning/SKILL.md b/plugins/aep/skills/planning/SKILL.md index 2394d60..44a39fb 100644 --- a/plugins/aep/skills/planning/SKILL.md +++ b/plugins/aep/skills/planning/SKILL.md @@ -108,7 +108,7 @@ of no scenarios or with no `spec_digest`. A report/2, which ESS writes on That record is what moves an `executable-system-specification` to `conforming` — its ladder is `draft → validated → conforming` — and `aep plan artifact set --model-digest ` ties it to the model the suite ran against; any other kind refuses the key by name. The -`ess:specifying` skill says how the report is produced — the `ess` plugin from the `beyond10x/ess` marketplace, or `ess skill specify` from the binary. +`ess:specifying` skill says how the report is produced (`b10x skill ess:specifying` prints it where the `ess` plugin is not loaded). **2. Every store mutation uses `aep plan artifact`; never edit a store file directly.** Creation, relations, status, and prose use `new`, `relate`/`unrelate`, `move`, and `body` respectively. **A @@ -127,9 +127,10 @@ template.** `aep plan artifact new` writes the kind's template into the store, a archive is still an artifact: it stays in the store, and its `move --to archived` is a lifecycle move a checker reads as work scheduled before the open question was filed. The complete example file is in [references/store-conventions.md](references/store-conventions.md), and `aep plan artifact new ---help` lists every flag. **A body file for `--from` goes under `$TMPDIR`, never a hard-coded -`/tmp`.** The runner sets `TMPDIR` to a directory it owns and reads back; `/tmp` is outside every -record it keeps. +--help` lists every flag. **A body file for `--from` is a scratch file inside the repository, in a +git-ignored directory** (for example `.engineering/drafts/`, added to `.gitignore`), or standard +input (`--from -`). Not a hard-coded `/tmp`, and not `$TMPDIR` either, which may point outside the +repository and outside a sandbox the session was confined to. ```console $ aep plan artifact body story:credential-store --from story-body.md @@ -209,7 +210,7 @@ means something beside `blocks:`, and `validate` says so. names an entity no ESS document in the repository declares (`ess/1`, or `ess/2`), do not decompose it and do not write stories around it. Draft the domain first — `aep plan reverse openapi --domain --out ` where an OpenAPI document already describes it, otherwise the minimal document in the -`ess:specifying` skill (`ess skill specify` prints it) — run `ess specify validate --path `, and cite the file by path in the +`ess:specifying` skill (`b10x skill ess:specifying` prints it) — run `ess specify validate --path `, and cite the file by path in the artifact body through `aep plan artifact body`. A noun with no typed home is the relation nobody can check later. @@ -328,7 +329,21 @@ $ aep plan reverse scan --format json ``` `reverse init` writes `.engineering/project.yaml` and refuses the two things that quietly break -later — an absolute path, and a `git+` source pinned to a branch rather than a commit. `reverse scan` +later — an absolute path, and a `git+` source pinned to a branch rather than a commit. + +The two values, for a project that follows the published protocols: + +| flag | value | +|---|---| +| `--protocols` | `git+https://github.com/beyond10x/aep#`, where `` is the commit of the release tag matching the installed `aep` (command below) | +| `--profile` | `development.standard`, unless the repository already names another | + +```console +$ git ls-remote https://github.com/beyond10x/aep "refs/tags/$(aep --version | awk '{print $NF}')^{}" +``` + +Where a backlog exists, `reverse init` is still the command that creates the store; `aep:migrating` +then moves the backlog into it. `reverse scan` reads and interprets nothing: it emits located facts — README headings, marked lines, tests that say they will not run, CI jobs and the variables on them, task targets, packages, published contracts — each carrying the `path:line` it was read from. It writes nothing and it has no clock and no network, @@ -460,7 +475,10 @@ created review-result:acceptance-round-1 (active) at .engineering/planning/revie you have recorded your reading of the review rather than the review. That includes the fenced ` ```findings ` block the rubric has each critic close with: it is the half a program reads, and a record whose findings were flattened into prose is one `aep plan artifact findings` cannot compare - against the next round. + against the next round. On `approve` the rubric's block is `[]`, which the store accepts. The one + edit allowed: a critic that closed with an empty fence (no `[]` inside) is recorded with `[]` in + it, because the store refuses the empty fence ("a review with nothing to report writes no block + at all"). Say in the report that you made that edit. * Repeat `--relate` once per artifact the critic judged. Read the edge name from `aep plan artifact relations` before you rely on it, the way you would any other vocabulary. * Write them one at a time. Four critics return at once; the store takes one writer. diff --git a/plugins/b10x/.claude-plugin/plugin.json b/plugins/b10x/.claude-plugin/plugin.json index 2b6eb23..141a6fb 100644 --- a/plugins/b10x/.claude-plugin/plugin.json +++ b/plugins/b10x/.claude-plugin/plugin.json @@ -2,7 +2,7 @@ "name": "b10x", "displayName": "Beyond10x", "description": "Set up, upgrade and check the Beyond10x plugins and binaries, route work to them, and create portable plugins.", - "version": "0.14.2", + "version": "0.14.3", "author": { "name": "Beyond10x" }, diff --git a/plugins/b10x/.codex-plugin/plugin.json b/plugins/b10x/.codex-plugin/plugin.json index 3e3ea12..5d5dc46 100644 --- a/plugins/b10x/.codex-plugin/plugin.json +++ b/plugins/b10x/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "b10x", - "version": "0.14.2", + "version": "0.14.3", "description": "Set up, upgrade and check the Beyond10x plugins and binaries, route work to them, and create portable plugins.", "author": { "name": "Beyond10x" diff --git a/plugins/b10x/skills/init/SKILL.md b/plugins/b10x/skills/init/SKILL.md index aabeabd..c77441c 100644 --- a/plugins/b10x/skills/init/SKILL.md +++ b/plugins/b10x/skills/init/SKILL.md @@ -28,7 +28,9 @@ Ask **one** multi-select question: *What do you want to do with Beyond10x?* | Work in isolated Git checkouts that clean up safely | `worktree` | | Connect external tools and providers | `connectors` | -Preselect what the user already named ("I want to write specs" → specifications). Use the host's +Preselect what the user already named ("I want to write specs" → specifications). When planning is +chosen, preselect specifications too and say why: AEP planning models every new noun as an ESS +domain before writing stories around it, so `aep` without `ess` stops at the first new entity. Use the host's question tool when it has one (Claude Code: `AskUserQuestion` with `multiSelect: true`). ## 3. Ask how to install the CLIs diff --git a/plugins/b10x/skills/routing/SKILL.md b/plugins/b10x/skills/routing/SKILL.md index 8abbd55..ce1d210 100644 --- a/plugins/b10x/skills/routing/SKILL.md +++ b/plugins/b10x/skills/routing/SKILL.md @@ -27,8 +27,10 @@ Route the request; do not reproduce a specialist plugin's full workflow. | Create, update, review, or port an installable plugin | `b10x:authoring-plugins` | | Plan or decompose work, review a plan, or reverse-engineer a backlog | `aep:planning` | | Scope and deliver accepted development work through a reviewed wave | `aep:implementing` | -| Specify, retrofit or conformance-test an ESS model | `ess` (from the `b10x` marketplace, released with the `ess` binary; `ess skill` prints the same skills) | -| Create, inspect, finish, or safely clean Git worktrees | `worktree` (from the `b10x` marketplace, released with the `worktree` binary) | +| Specify a system or API | `ess:specifying` | +| Derive a specification for an existing system | `ess:retrofitting` | +| Run or raise a conformance suite | `ess:testing-conformance` | +| Create, inspect, finish, or safely clean Git worktrees | `worktree:managing-worktrees` | | Set up providers, inspect Connector readiness, or invoke configured integrations through the CLI | `connectors:integrating` | ## Preserve boundaries diff --git a/plugins/connectors/.claude-plugin/plugin.json b/plugins/connectors/.claude-plugin/plugin.json index 37924fa..070c165 100644 --- a/plugins/connectors/.claude-plugin/plugin.json +++ b/plugins/connectors/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "connectors", - "version": "0.14.2", + "version": "0.14.3", "description": "Set up, inspect, and invoke governed integrations through the connectors CLI.", "author": { "name": "Beyond10x" }, "license": "Apache-2.0", diff --git a/plugins/connectors/.codex-plugin/plugin.json b/plugins/connectors/.codex-plugin/plugin.json index 0d4855b..55d83cb 100644 --- a/plugins/connectors/.codex-plugin/plugin.json +++ b/plugins/connectors/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "connectors", - "version": "0.14.2", + "version": "0.14.3", "description": "Set up, inspect, and invoke governed integrations through the connectors CLI.", "author": { "name": "Beyond10x" }, "license": "Apache-2.0", diff --git a/plugins/ess/.claude-plugin/plugin.json b/plugins/ess/.claude-plugin/plugin.json index 7f8649d..5e90774 100644 --- a/plugins/ess/.claude-plugin/plugin.json +++ b/plugins/ess/.claude-plugin/plugin.json @@ -2,7 +2,7 @@ "name": "ess", "displayName": "ESS", "description": "Write, retrofit, validate and project Executable System Specifications, and hold implementations to them with conformance suites.", - "version": "0.14.2", + "version": "0.14.3", "author": { "name": "Beyond10x" }, diff --git a/plugins/ess/.codex-plugin/plugin.json b/plugins/ess/.codex-plugin/plugin.json index 3a2355b..76bd2aa 100644 --- a/plugins/ess/.codex-plugin/plugin.json +++ b/plugins/ess/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "ess", - "version": "0.14.2", + "version": "0.14.3", "description": "Write, retrofit, validate and project Executable System Specifications, and hold implementations to them with conformance suites.", "author": { "name": "Beyond10x" diff --git a/plugins/ess/skills/retrofitting/SKILL.md b/plugins/ess/skills/retrofitting/SKILL.md index 6d966f2..2cd1085 100644 --- a/plugins/ess/skills/retrofitting/SKILL.md +++ b/plugins/ess/skills/retrofitting/SKILL.md @@ -1,7 +1,7 @@ --- name: retrofitting description: >- - Derive an ESS specification for a system that already exists and has none — from its OpenAPI contract, its observed Kubernetes deployment, or its code — then validate it and hand it to conformance. Use when the user asks to retrofit, adopt, reverse-engineer or "get a spec out of" an existing service or repository, when a codebase has an OpenAPI document or a cluster but no `system.yaml` or `ess-inputs.yaml`, or when a specification must describe behaviour that is already shipped rather than behaviour still to be built. Not for a domain drafted from nothing, which is the `specify` skill; not for raising coverage of a suite that already exists, which is the `coverage` skill. + Derive an ESS specification for a system that already exists and has none — from its OpenAPI contract, its observed Kubernetes deployment, or its code — then validate it and hand it to conformance. Use when the user asks to retrofit, adopt, reverse-engineer or "get a spec out of" an existing service or repository, when a codebase has an OpenAPI document or a cluster but no `system.yaml` or `ess-inputs.yaml`, or when a specification must describe behaviour that is already shipped rather than behaviour still to be built. Not for a domain drafted from nothing, which is the `ess:specifying` skill; not for raising coverage of a suite that already exists, which is the `ess:testing-conformance` skill. --- # Retrofitting a specification onto an existing system @@ -22,6 +22,10 @@ cites it. What you cannot read is an `UNMAPPED:` marker, never a plausible value Use every source that exists. Where two disagree, the draft records both and marks the field `UNMAPPED:`; the disagreement is a finding, not something to resolve by choosing. +`ess infra import openapi` reads OpenAPI 3.1 only. A 3.0 document exits 1 with the refusal +`only OpenAPI 3.1 is supported, found 3.0.x`; `aep plan reverse openapi` accepts it. Do not +rewrite the contract's version line to get past the refusal. + `aep` is optional. Without it, write the domain by hand from the schemas, in the shape the `ess:specifying` skill shows. @@ -51,7 +55,13 @@ Retrofit-specific rules: - **Relations need an owner decision.** `owns` against `references` is decided by what a delete does in the code today. No delete path found: `UNMAPPED:`. - **Views follow reads.** One view per read operation clients use; its fields are the fields the - response carries. + response carries. An entity with `invariants` also needs a view holding every state, or the suite + refuses the invariant checks (`ess:specifying`, conformance section). +- **Wire values keep their spelling in a comment.** A stored status `in_transit` becomes the state + `InTransit`; state names must start upper-case. +- **A rule the language cannot hold stays in the code, and is named.** A limit read from stored + state, or a constraint across records, is `UNMAPPED:` with its source line + ([syntax reference](../specifying/references/syntax.md), "What `when` can and cannot say"). ## 3. Prove the draft describes the system diff --git a/plugins/ess/skills/specifying/SKILL.md b/plugins/ess/skills/specifying/SKILL.md index 1e51077..1572e11 100644 --- a/plugins/ess/skills/specifying/SKILL.md +++ b/plugins/ess/skills/specifying/SKILL.md @@ -90,6 +90,10 @@ takes is refused as `missing_causation` — so a first domain has one state and gains a second state only together with the outcome that moves it. The header's `domains:` list and the declaring source must agree in both directions; either half alone is a refusal. +**Lifecycle state names start with an upper-case letter** (`InTransit`, not `in_transit`); a +lower-case one is refused as `invalid state name identifier`. A source that stores `in_transit` +still maps onto `InTransit` — note the mapping beside the state. Enum variants carry no such rule. + **A relation is an entry, not a convention.** Two entities linked by a field that happens to be named after the other one are not related as far as anything can check; the `relations:` entry is what makes the link a fact a program reads. Three more refusals come with it, and they are the @@ -113,10 +117,24 @@ Grow it from there — types, commands, events, views, and a component that owns shows every one of those sections in a small specification that validates; read it before writing the first command. -`--path` takes one ESS file or a directory. A directory is read through its `ess-inputs.yaml` when -one exists — an exact file list, which is what lets authored inputs sit in nested directories beside -generated files — and otherwise through the `system.yaml` layout above, which the CLI -calls legacy and still accepts. +`--path` takes one ESS file or a directory. Without an `ess-inputs.yaml`, a directory is read as +every YAML file below it — so generated output written inside it is read back as specification and +refused (`unknown field openapi`). **Add an `ess-inputs.yaml` as soon as anything is generated**, and +write generated output outside the specification's own files: + +```yaml +# ess-inputs.yaml, beside system.yaml +format: ess-inputs/1 +specification: + - system.yaml + - components.yaml + - domains/rooms.yaml +scenarios: [] +``` + +`specification` lists every authored file, relative to this file, and each must exist. `scenarios` +lists authored scenario files, `[]` when there are none. `--path ` then reads exactly +that list. The CLI help calls the list-less `system.yaml` layout "legacy"; it still validates. **A draft is a proposal, never a silent completion.** Every relation you could not read from code, an OpenAPI document or an existing artifact is written with an `UNMAPPED:` marker beside the place @@ -163,6 +181,10 @@ ess generate --path --kind schema --out ess generate project openapi --path --out ``` +Each writes into its own subdirectory of `--out`: `--out generated` gives `generated/schema/…` and +`generated/openapi/…`. Pass the parent, not `generated/schema`, or the files land in +`generated/schema/schema/`. + The same typed IR must produce the same ordered files and bytes. Compare a regenerated temporary tree with the committed tree before replacing anything. A stale committed file is drift; a file no projection owns is not authority. @@ -181,6 +203,12 @@ aep plan artifact evidence --from `synthesize` writes the suite the specification obliges — nothing for a domain with no commands or outcomes, so the two-entity draft above yields `0 scenario(s)` and no report worth recording. +An entity `invariant` is checked after every outcome that leaves the entity in some state, which +needs a view that holds instances in that state and publishes the fields the invariant reads. +Without one, `synthesize` refuses the check (`no view of holds an instance in `, or +` is published by no view of the entity`); a view over the entity with no `filter` that +publishes those fields covers every state (the `Copies` view in the syntax reference). `--target go` or `--target typescript` writes the suite as a test +package your implementation runs; `ess:testing-conformance` says what to run it against. `run` holds a built-in reference implementation to it; `--target` lists the ones this binary carries. `evidence --from` reads the kind, the source and the instant out of the report and refuses a report of no scenarios or with no `spec_digest`. A report/2 (`--report-format 2`) is diff --git a/plugins/ess/skills/specifying/references/syntax.md b/plugins/ess/skills/specifying/references/syntax.md index 42c1745..21d8425 100644 --- a/plugins/ess/skills/specifying/references/syntax.md +++ b/plugins/ess/skills/specifying/references/syntax.md @@ -289,11 +289,42 @@ views: naming: wire: available display: Available copies + + # An invariant is checked after every outcome, through a view that holds the entity in the + # resulting state and publishes the fields the invariant reads. With no `filter`, this one + # holds every state; without it, `ess verify conform synthesize` refuses the `pages > 0` checks. + - name: library.lending.Copies + source: library.lending.Copy + consistency: read_your_writes + fields: + - name: copy_id + type: library.lending.CopyId + - name: pages + type: Integer + naming: + wire: copies + display: Copies ``` +State names start with an upper-case letter (`OnLoan`); `validate` refuses `on_loan`. Enum +variants may be written as the source spells them. + ## What `when` can and cannot say A `when` reads only the command's own input. A condition on another entity — "the branch must be open", "the customer is active" — is not an input guard. Express it as a transition of that entity (a command that `moves` it, answered by `wrong_state` from states it does not start from), or leave an `UNMAPPED:` marker naming the rule and report it; never invent an outcome the compiler cannot decide. + +Two cases trials hit: + +| rule | how to write it | +|---|---| +| a value stored on the entity decides the outcome ("express parcels over 20 kg are refused at dispatch", with the weight given at create) | not expressible: a `when` sees only the dispatch input. Mark it `UNMAPPED:` at the outcome, citing the source line | +| two records must not overlap ("a room cannot be booked twice for one hour") | make the contested unit an entity with its own lifecycle (a `Slot` that is `Free` or `Booked`); a second booking is then `wrong_state` on that slot. Overlap between arbitrary time ranges is not expressible; mark it `UNMAPPED:` | + +**Order a time range by its length, not by comparing two `Timestamp`s.** `ends_at > starts_at` +validates, but `ess verify conform synthesize` refuses it (`no declared scale contains both +values`). `Duration` does not work either: it is text, and `validate` refuses `length > 0`. Take +`starts_at: Timestamp` and an `Integer` in a named unit (`duration_minutes`) and guard on +`duration_minutes > 0`. diff --git a/plugins/ess/skills/testing-conformance/SKILL.md b/plugins/ess/skills/testing-conformance/SKILL.md index 8d88a3b..9e0345d 100644 --- a/plugins/ess/skills/testing-conformance/SKILL.md +++ b/plugins/ess/skills/testing-conformance/SKILL.md @@ -194,12 +194,31 @@ most often assumed: Three consecutive runs with identical counts, and one run of the job's own script in the job's own image, before claiming a number. -## A new domain has no target yet +## Which target runs your suite -A suite synthesised for a domain nobody has implemented reports every scenario `unsupported`: none -of the built-in targets runs your domain, and that is expected until an implementation (or a -target wrapping one) answers `ExecuteCommand` and `QueryView`. Keep the suite; run it once there is -something to run it against. +`ess verify conform run --target` names a target built into the `ess` binary. None of them runs +your implementation: + +| `--target` | what it is | on your own domain | +|---|---|---| +| `interpreted` | a placeholder for running the specification itself; it decides nothing yet | every scenario `unsupported`, run `failed` | +| `oracle-fixture` | a hand-written implementation of ESS's own `examples/oracle-fixture` | every scenario `error` | +| `billing` | a hand-written implementation of ESS's own `examples/billing` | every scenario `error` | + +(Counts observed with a 21-scenario suite for a new domain.) They say nothing about your system, so +never record such a report as evidence. + +To hold your implementation to the suite, generate it as a test package in the implementation's +language and implement the package's `Target` interface over your service: + +```console +ess verify conform synthesize --path --target go --out +ess verify conform synthesize --path --target typescript --out +``` + +The package's `README.md` lists the methods and the wiring test. A method you cannot answer returns +`ErrUnsupported` (a skip), never a made-up result. Keep a suite for a domain nobody has implemented; +run it once something answers it. ## Agents diff --git a/plugins/worktree/.claude-plugin/plugin.json b/plugins/worktree/.claude-plugin/plugin.json index d2e2362..cd4aedd 100644 --- a/plugins/worktree/.claude-plugin/plugin.json +++ b/plugins/worktree/.claude-plugin/plugin.json @@ -2,7 +2,7 @@ "name": "worktree", "displayName": "Worktree", "description": "Create, lease, finish, audit and safely clean isolated Git worktrees through the worktree CLI.", - "version": "0.14.2", + "version": "0.14.3", "author": { "name": "Beyond10x" }, diff --git a/plugins/worktree/.codex-plugin/plugin.json b/plugins/worktree/.codex-plugin/plugin.json index 049540d..7eeacf6 100644 --- a/plugins/worktree/.codex-plugin/plugin.json +++ b/plugins/worktree/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "worktree", - "version": "0.14.2", + "version": "0.14.3", "description": "Create, lease, finish, audit and safely clean isolated Git worktrees through the worktree CLI.", "author": { "name": "Beyond10x" diff --git a/plugins/worktree/skills/init/SKILL.md b/plugins/worktree/skills/init/SKILL.md index 18f5081..b1643ae 100644 --- a/plugins/worktree/skills/init/SKILL.md +++ b/plugins/worktree/skills/init/SKILL.md @@ -22,15 +22,28 @@ No `b10x`? Follow https://github.com/beyond10x/agentplugins/releases/latest/down ## 2. First step here -Activate a workspace profile once, then check the setup: +Activate a workspace profile once, then check the setup. The profile is a small template; the +worktree repository publishes a default one: ```bash -worktree activate --profile --workspace +mkdir -p ~/.config/worktree +curl -fsSL -o ~/.config/worktree/default.toml \ + https://raw.githubusercontent.com/beyond10x/worktree/main/profiles/default.toml +worktree activate --profile ~/.config/worktree/default.toml --workspace worktree doctor --check ``` -`worktree doctor --check` names anything missing. Ask the user for the workspace root (the directory -that holds their repositories) rather than guessing it. +`activate` copies the profile into `~/.config/worktree/config.toml`, so the template file may stay +where it is or be committed into a repository the user shares with others. Ask the user for the +workspace root (an absolute path to the directory that holds their repositories) rather than +guessing it. + +`worktree doctor --check` exits 0 even when no profile is active; read its `profiles=` line and +treat `profiles=0` as not set up. + +**A repository needs a remote before its trees can be cleaned up.** Cleanup proves that each +commit reached a remote, so in a repository with no remote (`git remote` prints nothing) `create` +works and every later `gc` refuses. Tell the user before they start work there. ## 3. Pick the work diff --git a/plugins/worktree/skills/managing-worktrees/SKILL.md b/plugins/worktree/skills/managing-worktrees/SKILL.md index c461bbc..a34a3d0 100644 --- a/plugins/worktree/skills/managing-worktrees/SKILL.md +++ b/plugins/worktree/skills/managing-worktrees/SKILL.md @@ -10,7 +10,7 @@ Use the `worktree` CLI as the sole owner of linked-worktree lifecycle. It keeps ## Start repository work 1. From a primary checkout run `worktree create --purpose `. Add `--repo `, `--base `, or `--id ` when needed. -2. Treat the printed path as the task checkout and do all changes there. +2. Treat the printed path as the task checkout and do all changes there. The tree starts on a detached HEAD; run `git switch -c ` in it before the first commit, so the work has a branch to push. 3. If already inside a managed tree, reuse it; do not nest another worktree. 4. For automation, add `--json` and consume the versioned output. diff --git a/website/docs/structure.md b/website/docs/structure.md index 55a4e18..395dbf5 100644 --- a/website/docs/structure.md +++ b/website/docs/structure.md @@ -17,7 +17,7 @@ tools` checks the skills against the newest CLI releases. A change that breaks a | **R5 content** | A skill describes its CLI's newest release and quotes no CLI version. `agentplugins-check tools` runs every spelled command against that release. | | **R6 distribution** | `SETUP.md` and `b10x` install everything; CLIs come prebuilt or from `cargo`. Retired names live only in `catalog.json`, and setup migrates them. | | **R7 references** | Every `:` written in this repository names a file that exists. | -| **R8 docs** | One README row, one page under `plugins/` and one sidebar entry per plugin. The README is one paragraph and that table. | +| **R8 docs** | One README row, one page under `plugins/` and one sidebar entry per plugin. The README is one paragraph, that table, and the generated tree of every skill and agent, each linked to its file. | ## The plugins