Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .engineering/planning/story/ess-tutorial-and-onboarding.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
format: aep.planning-md/3
id: story:ess-tutorial-and-onboarding
kind: story
status: active
title: A public tutorial takes a developer from nothing to a conforming ESS specification; onboarding pages match b10x
summary: New tutorials/first-ess-specification page recorded with real output, held to the newest CLIs by the checker and a trial; onboarding pages corrected; released as 0.17.0.
relations:
- informed_by: epic:ahead-of-the-alternative
scope:
- confidence: cited
path: .github/workflows/tools.yml
- confidence: cited
path: README.md
- confidence: cited
path: SETUP.md
- confidence: cited
path: b10x.docs.yaml
- confidence: cited
path: crates/agentplugins-check
- confidence: cited
path: plugins/ess
- confidence: cited
path: trials/ess-tutorial
- confidence: cited
path: verified.json
- confidence: cited
path: website/docs
revision: 12
transitions:
- {from: "draft", to: "proposed", at: "2026-09-28T09:14:02Z", actor: "human:timo", revision: 11}
- {from: "proposed", to: "active", at: "2026-09-28T09:14:02Z", actor: "human:timo", revision: 12}
---
# Story: a public ESS tutorial, and onboarding pages that match b10x

## Outcome
A developer who reads the agentplugins docs can go from nothing to a validated ESS specification
and a passing Go conformance run, with an agent doing the writing, and every step shows the output
it really printed. The onboarding pages name the five plugins that ship and install them the way
`b10x` does.

## Context
Surveyed on `origin/main` d6c2ae1 (2026-09-28):
- No page in agentplugins, ess or aep walks from zero to a conformance run with real output; ESS
`getting-started.md` stops at the built-in `billing` target.
- `website/docs/install.md:39-77` hand-installs AEP 0.55.0 and expects `protocol 0.55.0`;
`golden-path.md:28` installs the `ess` plugin "from the ESS repository".
- `trust-and-scope.md:12-16` and `choose-a-plugin.md:12,32,50` name retired plugins ("AEP Plan",
"AEP Drive", "ESS Specify", "Beyond10x"); the retired-name check matches hyphenated ids only
(`crates/agentplugins-check/src/main.rs:332-453`).
- connectors is missing from `intro.md`, the docusaurus footer and `src/pages/index.tsx`.
- ess 0.38.0 is out and `verified.json` pins 0.37.0, so the Tools check fails on `main`.

## Acceptance
- `website/docs/tutorials/first-ess-specification.md` exists, recorded with ess 0.38.0; its
committed specification validates, and `agentplugins-check tools` validates it and checks every
command the page spells against the newest releases.
- `trials/ess-tutorial`: a fresh isolated agent given only the page reaches `validate: valid` and 0
failed scenarios under `go test`.
- `agentplugins-check` refuses the retired display names in `website/` and `README.md`.
- `task check`, `task site-build` and `tools` pass; release 0.17.0 publishes its assets.

## Out of Scope
The AEP tutorial (later); the `website` repository's `experiences.json`; the ess repository's
own guides.
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,30 @@
# Changelog

## [0.17.0] — 2026-09-28

A tutorial takes a developer from an empty directory to a validated ESS specification and a Go
implementation that passes its conformance suite, with an agent doing the writing. The onboarding
pages install through `b10x` and name the five plugins that ship. The skills are verified against
ess 0.38.0.

- New page: [Your first ESS specification](website/docs/tutorials/first-ess-specification.md).
`ess:specifying` interviews, writes a lending-library specification, and `ess` validates it,
refuses a planted mistake, generates docs and OpenAPI, and synthesizes a Go suite of 17
scenarios that the implementation passes and a planted bug fails. Every output is from a real
run. `agentplugins-check tools` validates the page's specification against the newest `ess` and
runs its suite, and the `ess-tutorial` trial has a fresh agent follow the page.
- `install.md` installs every command-line tool through `b10x init` and `b10x setup apply`, and
Metaharness through `b10x install metaharness`; the hand install of AEP 0.55.0 is gone.
`choose-a-plugin.md`, `trust-and-scope.md`, the b10x reference, the golden path's prerequisites,
the landing page, the footer and `b10x:routing`'s resources name `b10x`, `aep`, `ess`,
`worktree` and `connectors`.
- `agentplugins-check` refuses the retired display names "AEP Plan", "AEP Drive" and "ESS Specify".
- `verified.json` pins ess 0.38.0. From the trial round: `ess:specifying` counts "make sure it
validates" as a request for a finished specification and allows a named refusal where synthesis
cannot arrange the model; `ess:testing-conformance` generates the Go package at the module root,
requires `ESS_REPORT_FORMAT=2`, and has a refused command set `Outcome` (beyond10x/ess#186).
- This repository's planning store is `aep.project/5`.

## [0.16.0] — 2026-09-28

The `ess`, `aep` and `worktree` plugins describe the newest CLI releases: ess 0.37.0, aep 0.63.1
Expand Down
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ resolver = "2"
members = ["crates/agentplugins-check", "crates/b10x"]

[workspace.package]
version = "0.16.0"
version = "0.17.0"
edition = "2021"
rust-version = "1.85"
license = "Apache-2.0"
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ Most plugins drive a CLI of the same name. To have an agent install plugins and
migrate older installs, tell it: *"Set up Beyond10x: follow
https://github.com/beyond10x/agentplugins/releases/latest/download/SETUP.md"*.

New to ESS: [your first ESS specification](https://beyond10x.github.io/docs/agentplugins/tutorials/first-ess-specification/),
a tutorial from an empty directory to a passing conformance suite.

More: [install guide](website/docs/install.md) · [evals](evals/README.md) ·
[changelog](CHANGELOG.md) · [contributing](AGENTS.md)

Expand Down
4 changes: 4 additions & 0 deletions SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,7 @@ says so. Change nothing the user did not confirm.
3. **Continue with the product.** A plugin loads in the next session. To use it now, run
`b10x skill <plugin>:init` (for example `b10x skill ess:init`) and follow the printed text;
`b10x skill <plugin>` lists the rest.
4. **Point a newcomer at the tutorial.** When the user chose to write specifications and has not
written one before, tell them about
https://beyond10x.github.io/docs/agentplugins/tutorials/first-ess-specification/: it walks
one specification from an empty directory to a passing conformance suite.
7 changes: 5 additions & 2 deletions b10x.docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,12 @@ surfaces:
- kind: quickstart
label: Install
url: https://beyond10x.github.io/docs/agentplugins/install/
- kind: guide
label: 'Tutorial: your first ESS specification'
url: https://beyond10x.github.io/docs/agentplugins/tutorials/first-ess-specification/
- kind: reference
label: Plugin reference
url: https://beyond10x.github.io/docs/agentplugins/plugins/beyond10x/
url: https://beyond10x.github.io/docs/agentplugins/plugins/b10x/
- kind: source
label: Source
url: https://github.com/beyond10x/agentplugins
Expand All @@ -84,4 +87,4 @@ surfaces:
order: 30
sidebar: autogenerated
root: website
summary: Navigate and create portable plugins, or install focused AEP planning, AEP delivery, and ESS specification guidance from the curated b10x marketplace.
summary: Set up the b10x plugins for AEP planning and delivery, ESS specification, worktrees and connectors, then write and test a first ESS specification with the tutorial.
47 changes: 47 additions & 0 deletions crates/agentplugins-check/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,24 @@ const RETIRED: &[Retired] = &[
new: "the b10x marketplace",
wire_next: &[],
},
// Display names of the retired plugins. Public pages kept them after the ids were swept,
// because the sweep matched only the hyphenated ids (found 2026-09-28 in trust-and-scope.md,
// choose-a-plugin.md and the b10x routing resources).
Retired {
old: "AEP Plan",
new: "aep@b10x",
wire_next: &[],
},
Retired {
old: "AEP Drive",
new: "aep@b10x",
wire_next: &[],
},
Retired {
old: "ESS Specify",
new: "ess@b10x",
wire_next: &[],
},
];

/// Directories never walked looking for a retired name, wherever they sit.
Expand Down Expand Up @@ -1547,6 +1565,35 @@ one product only: `b10x upgrade ess`
std::fs::remove_dir_all(&sandbox).expect("the sandbox is removable");
}

/// A retired plugin's display name on a public page fails like its id, and a longer word that
/// begins with it (`AEP Planning`) does not.
#[test]
fn a_retired_display_name_on_a_page_fails_the_check() {
let sandbox = std::env::temp_dir().join(format!(
"agentplugins-check-display-{}-{:?}",
std::process::id(),
std::thread::current().id()
));
let page = sandbox.join("website/docs/trust-and-scope.md");
std::fs::create_dir_all(page.parent().expect("a page has a directory"))
.expect("the sandbox is writable");
let name = ["ESS", "Specify"].join(" ");
std::fs::write(&page, format!("- {name} validates contracts.\n"))
.expect("the sandbox is writable");
let error = retired_names(&sandbox).expect_err("a retired display name must fail");
assert!(
error.contains("website/docs/trust-and-scope.md:1 names `ESS Specify`"),
"{error}"
);

let longer = ["AEP", "Planning"].join(" ");
std::fs::write(&page, format!("- {longer} is a topic, not a plugin.\n"))
.expect("the sandbox is writable");
retired_names(&sandbox).expect("a longer word is not the retired name");

std::fs::remove_dir_all(&sandbox).expect("the sandbox is removable");
}

/// The name a file is *filed under* is a reference too, and the sweep never looks at one.
///
/// [`retired_names`] reads bytes and never paths, so the leftover an incomplete `git mv` makes
Expand Down
98 changes: 96 additions & 2 deletions crates/agentplugins-check/src/tools.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@
//! R5), so the only thing that can drift is a product release renaming or removing a command the
//! skills still spell. This check downloads each product's newest release — the prebuilt archive,
//! checked against its `SHA256SUMS` — and runs `<cli> <subcommands> --help` for every command a
//! code span or code block in that plugin spells. ESS's syntax example must also still validate.
//! code span or code block in that plugin, or in a page under `website/docs/tutorials/`, spells.
//! ESS's syntax example must also still validate, and the public tutorial's specification must
//! validate, synthesize a Go suite with no refusals and pass it with its committed implementation.
//! It runs on every pull request, every `main` push and daily; a red run is fixed by a skill edit.
//!
//! It also fails when a CLI's newest release is newer than `verified.json`, the release its skills
Expand Down Expand Up @@ -309,6 +311,95 @@ fn syntax(root: &Path, ess: &Path, scratch: &Path) -> Result<(), String> {
Ok(())
}

/// The public tutorial's committed specification and Go implementation.
pub const TUTORIAL: &str = "website/docs/tutorials/first-ess-specification";

/// Tutorial pages, whose spelled commands are held to the newest releases like the skills'.
pub const TUTORIALS: &str = "website/docs/tutorials";

fn copy(from: &Path, to: &Path) -> Result<(), String> {
std::fs::create_dir_all(to).map_err(|error| format!("{}: {error}", to.display()))?;
for entry in std::fs::read_dir(from).map_err(|error| format!("{}: {error}", from.display()))? {
let path = entry.map_err(|error| error.to_string())?.path();
let target = to.join(path.file_name().unwrap_or_default());
if path.is_dir() {
copy(&path, &target)?;
} else {
std::fs::copy(&path, &target)
.map_err(|error| format!("{}: {error}", path.display()))?;
}
}
Ok(())
}

/// Run in `dir` with `ESS_TOOLCHAIN_DELEGATED=1`, so a `requires:` pin in the tutorial's
/// `ess-inputs.yaml` does not hand the command to the pinned release: the newest one is under test.
fn run_in(dir: &Path, program: &str, arguments: &[&str]) -> Result<String, String> {
let output = Command::new(program)
.args(arguments)
.current_dir(dir)
.env("ESS_TOOLCHAIN_DELEGATED", "1")
.env("ESS_REPORT_FORMAT", "2")
.output()
.map_err(|error| format!("running {program}: {error}"))?;
let text = format!(
"{}{}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
if output.status.success() {
Ok(text)
} else {
Err(format!(
"{program} {} failed: {}",
arguments.join(" "),
text.trim()
))
}
}

/// The tutorial, as a reader runs it: its specification validates and synthesizes a Go suite with
/// no refusals, and its implementation passes that suite under `go test`.
fn tutorial(root: &Path, ess: &Path, scratch: &Path) -> Result<(), String> {
let dir = scratch.join("tutorial");
copy(&root.join(TUTORIAL), &dir)?;
let ess = ess.to_string_lossy();
let fail = |step: &str, error: String| format!("{TUTORIAL}: {step}: {error}");
let validated = run_in(&dir, &ess, &["specify", "validate", "--path", "spec"])
.map_err(|error| fail("the specification no longer validates", error))?;
println!("tools `ess`: tutorial — {}", validated.trim());
let synthesized = run_in(
&dir,
&ess,
&[
"verify",
"conform",
"synthesize",
"--path",
"spec",
"--target",
"go",
"--out",
"impl",
],
)
.map_err(|error| fail("synthesize failed", error))?;
if !synthesized.contains(" 0 refusal(s)") {
return Err(fail(
"the specification synthesizes with refusals",
synthesized.trim().to_owned(),
));
}
println!("tools `ess`: tutorial — {}", synthesized.trim());
let tested = run_in(&dir.join("impl"), "go", &["test", "./..."])
.map_err(|error| fail("the implementation fails its suite", error))?;
println!(
"tools `ess`: tutorial — go test: {}",
tested.lines().next().unwrap_or_default().trim()
);
Ok(())
}

/// Check every plugin's spelled commands against its CLI's newest release.
pub fn verify(root: &Path) -> Result<(), String> {
let scratch =
Expand All @@ -326,7 +417,9 @@ pub fn verify(root: &Path) -> Result<(), String> {
problems.push(line);
}
let mut checked = 0;
for file in markdown(&root.join("plugins").join(plugin)) {
let mut files = markdown(&root.join("plugins").join(plugin));
files.extend(markdown(&root.join(TUTORIALS)));
for file in files {
let text = std::fs::read_to_string(&file).map_err(|error| error.to_string())?;
for path in spelled(&text, cli) {
checked += 1;
Expand All @@ -348,6 +441,7 @@ pub fn verify(root: &Path) -> Result<(), String> {
println!("tools `{cli}` {tag}: {checked} spelled command(s) checked");
if *cli == "ess" {
syntax(root, &binary, &scratch)?;
tutorial(root, &binary, &scratch)?;
}
}
if problems.is_empty() {
Expand Down
20 changes: 20 additions & 0 deletions crates/agentplugins-check/src/trials.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ pub enum Kind {
EssPipeline,
/// Every ESS output, and an implementation held to the synthesized Go suite.
EssFullPackage,
/// A fresh agent follows the public ESS tutorial, given only the page.
EssTutorial,
/// AEP planning from an existing backlog.
AepBacklog,
/// Setting up `worktree` in a repository.
Expand Down Expand Up @@ -403,6 +405,7 @@ mod tests {
"ess-retrofit",
"ess-pipeline",
"ess-full-package",
"ess-tutorial",
"aep-backlog",
"worktree-onboarding",
"upgrade-seeded",
Expand All @@ -411,6 +414,23 @@ mod tests {
}
}

/// The tutorial trial hands its agent a copy of the public page; a copy that differs would
/// measure a page nobody reads.
#[test]
fn the_tutorial_trial_carries_the_published_page() {
let root = repository();
let page =
std::fs::read_to_string(root.join("website/docs/tutorials/first-ess-specification.md"))
.expect("the tutorial page exists");
let copy =
std::fs::read_to_string(root.join(TRIALS).join("ess-tutorial/fixture/tutorial.md"))
.expect("the trial fixture carries the page");
assert!(
page == copy,
"trials/ess-tutorial/fixture/tutorial.md differs from the tutorial page; copy the page again"
);
}

#[test]
fn a_definition_that_does_not_hold_together_is_refused() {
let root = scratch("bad");
Expand Down
2 changes: 1 addition & 1 deletion plugins/aep/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "aep",
"displayName": "AEP",
"description": "Plan governed work in the AEP artifact store and deliver it in reviewed waves: decomposition, plan critique, reverse engineering, story scoping, implementation and adversarial review.",
"version": "0.16.0",
"version": "0.17.0",
"author": {
"name": "Beyond10x"
},
Expand Down
Loading
Loading