Skip to content

Learning tier item 1: the mentor writes (record_teachback tool, trigger table on all six harnesses, scripted-learner proof) #38

Open

Description

@NetDevAutomate

Why this is the MVP blocker

The mentor agent can read learning state on every harness but records nothing: no MCP teach-back writer exists (CLI only), the persona's only concrete "record progress" line points at a different tool, and no harness has a trigger table telling the mentor when to write. Measured result: zero writer calls in 143,973 historical messages. So the "single recommended next action" the README promises is fed by flashcard/quiz reviews and CLI writes — never by what happened in a mentor session. Owner verdict 2026-09-23: this is the failure that matters most in a stranger's hands.

Council-validated plan: docs/architecture/learning-tier/plan-2026-09-19.md (§3, item 1). Arbitration: docs/architecture/learning-tier/council/arbitration-plan-2026-09-19.md. Owner decisions 1–5 recorded in plan §7 (#34, PR #36).

Branch

feat/learning-tier-fed, cut from main when S1-0 starts (decision 1). Item 2 (the Jev measurement) is not on this branch and its branch is not cut until 0.5.x closes.

Stages (plan §3 — each has its own finish)

  • S1-0 Capability lock (½ day). Receipt docs/architecture/learning-tier/receipts/s1-0-capability-lock.md with file:line per harness: the grant grammar as it is; W_auto = log_topic, log_struggle, record_teachback (new), record_plan_learning — named in every definition, prompt-per-call everywhere, no new pre-approval (decision 2); W_srs stays prompt-per-call; OpenCode's wildcard recorded as known and out of scope.
  • S1-RED (1 day). Failing tests per the plan's §5 table: test_mcp_teachback.py (tool absent), test_adapter_parity.py (names absent; no new grants), test_docs_harness_tier_contract.py (trigger table absent; persona still routes "record progress" to tutor-checkpoint), test_writer_isolation.py (guard, not RED if it already passes), no-trigger/duplicate replay.
  • S1-GREEN (1–2 days). The record_teachback MCP tool validating exactly as cli/_teachback.py does; agents/shared/recording-protocol.md with the fenced YAML trigger table, projected byte-identically to all six harness definitions with manifest hashes; every definition names each W_auto tool (Claude's socratic-mentor.md tools: line names none today); persona's record-progress line repointed.
  • S1-SIM (1 day + harness logins — owner-gated). Scripted learner drives each installed harness through a teach-back episode; evidence bundle per harness; PR body lists which harnesses passed and names the skip reasons for the rest. Claim is exactly "pipe open on {passed}; plumbing proven for all six definitions; noticing observed once".
  • Noticing episode (observation, not gate — owner). By rule (decision 3): the first real study session after S1-GREEN reaches main, ten minutes, no logging asked for; date and adjudicated outcomes written into the S1-SIM receipt.

Definition of done (= plan §3 S1-SIM finish)

CI green (parity, contract, projection, isolation, replay); evidence bundles for the harnesses that ran; the claim stated as above; council implementation review (3 seats) before merge; full unit suite green; just lint; just typecheck. Closing this issue closes item 1 only; item 2 has its own issue on 0.6.0.

Activity

  1. added this to the 0.5.x milestone on Sep 23, 2026
  2. added
    ready-for-agentImplementation-ready specification with settled requirements and test seams
    on Sep 23, 2026
  3. NetDevAutomate commented on Sep 23, 2026

    @NetDevAutomate
    OwnerAuthor

    S1-0, S1-RED and S1-GREEN are on PR #41 (5670f941); council review 9 is recorded with GATE ACCEPT at 616fd1ed (docs/architecture/learning-tier/council/review-9-arbitration-2026-09-23.md).

    S1-SIM checklist (adopted from the grok seat, arbitration gate decision) — per harness that is to count as passed:

    1. Happy path teach_back_agreed: scores proposed → learner says yes → exactly one record_teachback call → one row → reply recorded: true.
    2. Refusal path: scores proposed → learner says no / low-energy blank → zero calls, zero rows. A harness that records on refusal is not in {passed}.
    3. The other three triggers once each (log_struggle, log_topic per confirmed concept, record_plan_learning only with an active plan and agreement; no plan → zero calls).
    4. No-trigger silence: a matching-nothing turn writes nothing (row counts before/after).
    5. Duplicate replay: two agreed teach-backs on one concept → two rows.
    6. Decision-2 live: on Kiro only log_topic is un-prompted; on Claude the four writers are callable and prompt; OpenCode MCP approval is the harness default.
    7. Claude reachability: the tools: line is proven live for the first time — highest-risk item.
    8. Grok projection: session-dir AGENTS.md is the built persona and the running mentor names the four writers.
    9. N1 live: teach_back_scores.session_id IS NULL, reply study_session_id equals the live session.
    10. Isolation under a real harness with HOME pointed at a decoy.

    Plus (astra Y3): whether a Codex / Claude Code / OpenCode mentor launched outside the repository resolves the agents/shared/… reference — pre-existing for all nine shared protocols, decided here.

    Owner steps: harness logins for the SIM run; the ten-minute noticing episode (decision 3) after S1-GREEN reaches main.

  4. NetDevAutomate commented on Sep 28, 2026

    @NetDevAutomate
    OwnerAuthor

    Reopened: this issue was closed by accident on 24 Sep, not because item 1 is finished.

    How it closed. GitHub records commit 6b6a35b ("Merge PR #41 ...", landed with #43) as the closer. Its message explained that #41's body had been changed to "Part of #38" so this issue would stay open, and to say so it quoted the old closing keyword with this issue's number, wrapped across a line break. GitHub's keyword matching ignores the quotation marks and the line break, so the sentence closed the issue it was written to keep open. GitHub also lists #41 and #43 as closing references: their bodies say "Does not close" and "no longer closes" followed by this issue's number, and the matching doesn't understand negation either. The owner decision is unchanged: S1-SIM and the noticing episode stay open here.

    Still open

    1. Web sessions don't get the recording instructions. Every web Study and Body Double session is built with persona mode focus (persona_mode_for, packages/studyloop/src/studyloop/agent_launcher.py:253). There is no agents/shared/personas/focus.md, so build_canonical_persona falls back to _default_persona (line 414): a generic text that names none of the four writers and tells the mentor to "drive the session". The S1-GREEN persona fix in personas/study.md therefore reaches studyloop study in a terminal only. For web Kiro sessions over ACP (agent studyloop, which carries no instructions of its own) the fallback is everything the mentor is told. test_focus_persona_unchanged pins the fallback, and the S1-GREEN persona test builds only study and co-study. To be fixed test-first.
    2. S1-SIM: the scripted-learner run with harness logins (checklist earlier in this issue).
    3. The ten-minute noticing episode (decision 3). Run in a browser before item 1 is fixed, it would test the fallback rather than S1-GREEN.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentImplementation-ready specification with settled requirements and test seams

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions