Studio: design system, reports, live views, Genesis as an autonomous scientist, identity file (features 007-020) - #2
Open
LucasWaki wants to merge 39 commits into
Open
Studio: design system, reports, live views, Genesis as an autonomous scientist, identity file (features 007-020)#2LucasWaki wants to merge 39 commits into
LucasWaki wants to merge 39 commits into
Conversation
…ce, regrade revisions, runtime manifests, Studio with product graphs, offline Enterprise adapter; unblock plan of 8 Sep Offline foundation work of feature 007 and the feature 011 checkpoints, as recorded in specs/007-lab-foundation/implementation.md and specs/011-monarch-runtime-integration/. Paid dispatch stays gated in the CLI. Frozen task sets untouched. Full suite: 1091 passed, 3 skipped. Also: docs/AI-LABS-UNBLOCK-PLAN-2026-09-08.md (the plan settled with Lucas), the lab direction and Studio review notes, research/ scaffolding (ledger database ignored), .gitignore entries for the unrelated files that landed in this checkout (ignored by name, not deleted). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… plan files, hashed when not the default Unblock plan M2 T2.3. The default keeps every existing plan hash, the same rule retry_on_fail follows. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…lder on a volume, host flag; Dockerfile and .railwayignore for Railway Unblock plan D4 and Lucas's request of 8 Sep (web app on Railway behind a login). STUDIO_AUTH_USER/PASSWORD turn on HTTP Basic Auth; STUDIO_PUBLIC_HOSTS names the hosted host; STUDIO_DATA_DIR moves jobs and the weekly ledger to the mounted volume. Binding to a non-local address without the auth pair is refused. 28 Studio tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…er API request, attempt cap, round admission, Monarch attempts reserve their ceiling Milestone M3 of the unblock plan, steps T3.1 and T3.2. The CLI's API loop reserves every provider request for its rate-card maximum, claims it, sends it and settles from the usage receipt; an unreadable receipt keeps the hold. Reservation ids are stable per run, attempt, invocation and turn, so a retry or a resume never reserves a settled request again. Plan field attempt_cap_usd (default 3.00, hashed only when set) ends an attempt as infra:attempt_cap, which resume treats as final. An exhausted week ends the attempt as infra:weekly_budget and stops the run with stop_reason weekly_budget, resumable when the week has room. Round admission refuses a round whose maximum liability (API attempts x cap + Monarch attempts x the Monarch ceiling, capped by cost_ceiling_usd) exceeds the week's capacity, naming the shortfall, before the run row exists; resume counts only the attempts left. Monarch attempts reserve MONARCH_ATTEMPT_CEILING_USD per attempt through the same ledger and settle from Langfuse; the ceiling constant moves to wb_arms.monarch and the Studio re-imports it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ly Railway branch, seeds pin the front-door host, hosted Studio Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…onstructed catalog it descends from Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…of the blanket refusal, doctor through the ledger, wb budget reconcile Milestone M3, steps T3.3 to T3.5. wb_orchestrator/approvals.py: the launcher is WB_OPERATOR (required for any paid launch); approvers default to lucas (WB_APPROVERS overrides). Above smoke scale an approver's wb run runs at once under an approved record (new results-store table approval_requests); anyone else's writes a pending request, prints "<id> awaiting approval" and exits 0. wb approve / wb deny (approvers only), wb approvals, and wb run --request <id> (any operator, config hash must match, single use). approved_by in plan files is optional and ignored with a one-line notice; the resolve-time gate is gone; wb monarch recipes proceeds on --yes only. approvals.launch_readiness replaces _paid_launch_blocked: API-loop competitors launch with an operator and the ledger (top-level wb --ledger, default research/budget.sqlite3); Monarch competitors, wb monarch recipes and wb doctor --monarch-probe are refused with "Monarch instance not verified: milestone M5", native competitors with "native runtime not verified: milestone M7". The doctor's provider probes reserve every request through the ledger. wb budget reconcile --week --provider --csv compares a week's normalized usage rows with the ledger's settled total per provider, writes research/reconciliation/<week>.md and .json, and marks historical_billing_verified for that week only when every provider with spend is within 5 %; wb budget status reports it per week with the capability matrix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rk skill; config README on the attempt cap, approvals, the ledger and wb budget reconcile Lucas approves paid rounds; a launch by Carlos creates a request and waits; the weekly ledger is the spending gate. The README explains attempt_cap_usd, the ignored approved_by key, the approval commands, round admission, and how to export the usage rows wb budget reconcile reads from the Anthropic Console, the OpenAI usage page, Fireworks, Google and Langfuse (for Monarch). The shipped plan files' approved_by comments now say the key is ignored (comments only; no hash moves). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s manifest; the achievable-50 id list (unblock plan M2, T2.1 and T2.2) The command copies each corpus task unchanged into the set's folder and writes <name>-manifest.yaml beside it (selection rule from the id list's comment lines, why the set exists, suite revision from the installed automation-bench, cut points reused from the tier manifest, count per domain, one row per task with domain, score, tier and hash). It refuses, naming every offender, when an id is missing from the corpus, has no approval rule, does not match its own hash, or already sits in a frozen set; --refreeze keeps the ids and refreshes the copies; --allow-frozen-overlap records an overlap instead of refusing. tasks/achievable-50-ids.txt lists the 50 ids of the ApplicationBench achievable50 slate with the rule and source above them. Running the command on it refuses today: operations.docusign_prospect_nda sits in tier-complex and support.reamaze_cross_platform_dedup in random-10 (drawn 4 Sep, seed 20260904), so the set is not frozen until Lucas decides. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…request) and achievable-50-workflow (track create-run) (unblock plan M2, T2.4) Same task set, mode, size, competitors (answer key, claude-opus-5/api as baseline, monarch-stock, monarch-lab), ceiling US$ 220 and internal audience; they differ only in name, description and track. The Monarch harness files arrive with M5, so the plans load but do not resolve yet; the claude-code competitor joins with M7. Tested through config.load_plan. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rack) and EXP-2026-002 (workflow track), ledger lines, and the two round docs (unblock plan M2, T2.5 and T2.6) Control claude-opus-5/api; treatment monarch-lab (secondary monarch-stock); primary comparison lab against bare on first-attempt strict pass rate, paired by task, McNemar as wb report computes; prediction at least 5 points from the 26 Aug 2026 record (403/600 against 348/600); ceiling US$ 220 reserved in the weekly ledger; promote to a replication on a fresh draw only with p < 0.05 and no integrity flag. Status hypothesis. Round docs in the shape of the tier docs; status: set listed, freeze pending Lucas's decision on two ids already in frozen tier sets, re-freeze pending the evalrepair.10 import, Monarch harness files pending M5. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rlap with two tier ids recorded); re-freeze pending the evalrepair.10 import Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…y) on both tracks, runnable before the Monarch instances exist Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…orld; vendor script with a version check and a provenance record scripts/vendor_automation_bench.py copies a source tree into vendor/automation-bench, refuses any version other than --expect-version, leaves .venv, caches, logs and .git behind, and writes VENDORED-FROM.txt (source, version, pyproject hash, file count, content hash, tree id as given, date). pyproject pins automation-bench==1.0.6+evalrepair.10; uv.lock moved on that one package (uv 0.9.10 also rewrote two platform markers). Milestone M1 of docs/AI-LABS-UNBLOCK-PLAN-2026-09-08.md, T1.1. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… wb run refuses a set imported under another world; reports never pool suites wb corpus import-ab --revision LABEL --out DIR stamps info.world (package, installed version, label; hashed) and writes DIR/MANIFEST.yaml; wb corpus manifest rewrites it with per-domain counts, usable counts and the tasks without a rule. suite_id(tasks) gives workflowbench-synthetic@<version> for such a set and keeps @0.1 for the old ones; the orchestrator and the Studio record it on runs and rows, resume refuses a suite drift, and wb summary refuses rounds of different suites. config.resolve refuses a task set whose recorded world (1.0.6 when none) is not the installed package, naming both. seeded_services() restores what 'services seeded' meant now that the repaired world spells out all 48 apps' empty defaults in every scored task: declare's side-effect allowance, the difficulty measure and the two product service checks use it. Milestone M1, T1.2 to T1.4. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…lrepair10 (1.0.6+evalrepair.10), approval rules derived, manifest written 800 tasks (simple 200; finance, hr, marketing, operations, sales, support 100 each), 0 unmapped assertion types, 800 usable. Every task carries info.world; corpus/ (upstream 1.0.6) is untouched as the record of the old world. Milestone M1, T1.2. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Source, versions, hashes, commands, test counts before and after, the world's effect on the bench (materialized defaults, the 48th app, the no-op validator) and what was left for M2 and M5. Force-added: .gitignore line 64 (commit e033c90) ignores /specs/007-lab-foundation/, which T0.1 should undo. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…th descriptions from the PG-Waki catalog through an explicit table Unblock plan of 8 Sep 2026, M6 T6.3 groundwork. `wb monarch knowledge --knowledge <catalog.json> --out <dir>` generates the stock seeds and enriches `business_action.description` of every matched action with the reviewed catalog's purpose, non-effects, idempotency, record location (only where the seed's own response schema reaches it) and argument meaning (only for parameters the seed has); each product's paragraph opens its first action prefixed "Product:" since `_meta.json` has exactly five keys. Ids, verbs, url templates, parameters, extracts, schemas and `_meta.json` are byte-identical to the stock set; Monarch's validator reports the same 0 errors and 170 warnings on both. The catalog's ids are AutomationBench Zapier tools (`zapier:<app>_<tool>`) and the seeds' are REST routes (`bench-<app>:<verb>:<object>`); no rule derives one from the other, so the mapping is the explicit table `config/products/simulated-apps.knowledge-map.yaml` (257 rows, 78 notes), read from the data product by product. `<dir>/KNOWLEDGE-MAPPING.yaml` lists the catalog's sha256, the rule, the counts, every pair, the 16 entries without a row (with reasons) and the 462 actions without an entry; `ok.txt` carries `knowledge_sha256` and `knowledge_source` next to the seed version. Same inputs, same bytes. `wb monarch setup --knowledge` runs the same step before validating, checking and importing, and records the two fields in the knowledge-base hash file. Nothing is imported anywhere by this change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… lab seed set `KNOWLEDGE-MAPPING.yaml` as written by `wb monarch knowledge` on 8 Sep 2026 from catalog sha256 7aea3d99... and the shipped table: 273 entries, 257 matched onto 224 of the 686 bench actions, 16 catalog-only (each with its reason), 462 bench-only, 43 product paragraphs applied, 4 products without one. Seed folder digest dd03a8698bdb...; front door pinned to http://host.docker.internal:9105 for this offline run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…authoring backstops) into 007-benchmark-foundations Resolved: pilot plan ceiling US$ 5 with the approval note; deferred.md keeps both additions; orchestrator attaches the journal and sets the artifacts folder; the Monarch arm keeps the stream-then-poll follower and gains the no-writes verdict; the engine-terminal check is renamed _run_terminal so it no longer shadows the authoring terminal-frame handler; the orphan-deletion attribute is initialised in __init__; the SSE reader takes origin's raw-socket loop under the generic stream label. Monarch, Studio Enterprise, recipes and run-only tests: 114 passed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…stray files on 8 Sep) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…air10; world revision on every task; reports never pool suites) into 007-benchmark-foundations Resolved: the run start keeps the ledger admission and uses the set's own suite id. Vendored copy replaced with scripts/vendor_automation_bench.py; installed package automation-bench 1.0.6+evalrepair.10. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d table, mapping report) into 007-benchmark-foundations Resolved: .gitignore and deferred.md keep both sides' additions. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… (same 50 ids) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… flow, evalrepair.10 done; what is ready to trigger and what is not Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…anifest (re-freeze on another revision) Full suite on the merged tree: 1251 passed, 3 skipped, this one test failed and is fixed; test_slate 14 passed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… hosted front door; the three steps left for the Monarch arms Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d instance admits Monarch competitors for two hours Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ont door (47 apps, 8 Sep night) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…COMMIT) freezes its manifest on a hosted bench Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ilot monarch-pilot-001; how Monarch rounds launch Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…omous scientist, identity file (features 012-020) Studio: one token sheet over Radix Colors with IBM Plex and Lucide vendored, measures computed on the server (Wilson intervals, paired deltas, cost), run and round reports with caveats from data and a historical Bare baseline, the live product graph read from Feature Discovery over GET only, a Graph view per bench version, the streaming run page without duplication, the sticky inspector, the weekly budget chip, and the browser suite (27 checks). Genesis (feature 019): cards as inputs worked by a watcher under per-card and daily ceilings, a daily Monarch code index with read-only code tools, a bounded three-tier memory with nightly consolidation, a Trello-like board, a fixed library topic list, and the daily scheduler. Feature 020, first item: SOUL.md, the identity file. A starter text on first start, edited only by a person from the Memory tab, the first block of every prompt, out of reach of every Genesis tool. Also: Windows long-path handling in evidence writes and a short pytest temp root; wb studio port from the environment; research log, plan and handoff docs of 8 and 9 Sep. Ignored: local scratch, the Monarch checkout snapshot under artifacts, and Genesis test sessions. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… the public front door) into 007-benchmark-foundations
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
This branch carries the Studio work of 8 and 9 Sep on top of the benchmark foundations (features 007 to 011), and closes with feature 019 and the first item of feature 020.
Studio (features 012 to 018)
!important, inline style or external URL.Genesis (feature 019)
LAB.mdandMONARCH.mdat 2,500 characters each, notes per card, an FTS5 record, seven-day probation, thirty-day decay, pinned entries that never decay, a nightly consolidation turn under US$ 0.50.Feature 020, first item: the identity file
SOUL.mdholds Genesis's voice, priorities and what it never does. A starter text is written on first start. Only a person edits it, from the Memory tab. It is the first block of every prompt and no Genesis tool reaches it.Also
wb studiotakes its port from the environment.Why
The plan of 9 Sep (
docs/AI-LABS-IMPLEMENTATION-PLAN-2026-09-09.md) mapped the Studio into seven phases. The Genesis design follows the research notedocs/AI-LABS-GENESIS-CODE-AWARE-MEMORY-2026-09-09.md; Lucas's choices for what Genesis grows next are in its Part 3.For the reviewer
tests/test_run_config.py::test_shipped_create_run_pilot_hash_is_unchanged, which already failed atmainand waits on Lucas's sign-off to re-pin the hash (pre-registration rule).node tests/browser/suite.cjs, 27 checks pass. Screenshots undertests/browser/snapshots/are for review, not pixel diffing.main(handoff of 8 Sep, deferred item on the public front door) are merged in; the one conflict, indeferred.md, kept both sides..gitignore: the Monarch Enterprise checkout snapshot underartifacts/enterprise-local, Genesis test sessions underartifacts/genesis-check, and local scratch folders.PLAN.md§1. No task prompt, data or approval rule was edited.🤖 Generated with Claude Code