diff --git a/docs/README.md b/docs/README.md index 885fadc..7869e50 100644 --- a/docs/README.md +++ b/docs/README.md @@ -32,6 +32,8 @@ Start here: - [Scient product identity](product/scient-product-identity.md) - accepted company, app, native-agent, external-agent, and naming vocabulary. - [Product philosophy](product/product-philosophy.md) - draft durable product principles; the accepted PRD governs conflicts. - [Technology stack](architecture/technology-stack.md) - current proposed stack direction. +- [Idea inbox](planning/idea-inbox.md) - categorized intake for unresolved ideas, + including the future Scient memory-architecture discovery. - [Product roadmap](planning/product-roadmap.md) - current sequence of coherent product outcomes. - [First vertical-slice implementation plan](planning/first-scient-vertical-slice-implementation-plan.md) - bounded plan for the active product slice. - [Scient and external agents implementation plan](planning/scient-and-external-agents-implementation-plan.md) - proposed plan for building the Scient agent as the owned OpenCode-derived first-party agent while preserving external agents independently. diff --git a/docs/architecture/local-first-sync.md b/docs/architecture/local-first-sync.md index ba4ff3c..ebbf7cf 100644 --- a/docs/architecture/local-first-sync.md +++ b/docs/architecture/local-first-sync.md @@ -3,16 +3,20 @@ Status: Placeholder Owner: Yaacov Created: 2026-06-27 -Last updated: 2026-07-17 +Last updated: 2026-07-18 Purpose: Defines what should be documented about Scient local-first storage and cloud sync once the design is validated. Doc type: Future home -This page will document Scient's local-first and sync architecture when it exists. +This page will document Scient's local-first and sync architecture when it +exists. Unprocessed questions about memory scope, user-selected cloud folders, +offline behavior, conversation continuity, and future Scient cloud sync remain +in the [Idea Inbox](../planning/idea-inbox.md#memory-context-and-continuity). +No canonical memory store or sync engine is selected. Document here: - offline behavior -- local SQLite boundaries +- local application-state and project-owned-state boundaries - cloud mirror semantics - sync engine selection - conflict handling diff --git a/docs/architecture/project-format.md b/docs/architecture/project-format.md index 2694092..55382d0 100644 --- a/docs/architecture/project-format.md +++ b/docs/architecture/project-format.md @@ -3,11 +3,15 @@ Status: Placeholder Owner: Yaacov Created: 2026-06-27 -Last updated: 2026-07-17 +Last updated: 2026-07-18 Purpose: Defines what should be documented about Scient project structure once the format is designed. Doc type: Future home -This page will document the Scient project format when it exists. +This page will document the Scient project format when it exists. Unprocessed +questions about project memory, conversations, files, portability, Git, cloud +folders, and storage boundaries remain in the +[Idea Inbox](../planning/idea-inbox.md#memory-context-and-continuity) until a +dedicated memory-architecture discovery begins. Document here: diff --git a/docs/architecture/technology-stack.md b/docs/architecture/technology-stack.md index 2d08106..61afb23 100644 --- a/docs/architecture/technology-stack.md +++ b/docs/architecture/technology-stack.md @@ -62,11 +62,11 @@ Scient app from the planned Scient agent where needed. | Local coordinator | Bun/Node.js WebSocket server | Inherited scaffold candidate; not yet a Scient decision | | Workspace tooling | Bun workspaces, Turborepo, Vite | Inherited scaffold candidate; not yet a Scient decision | | Cloud web app | React, with Next.js as a later candidate | Not scaffolded | -| Local database | SQLite | Proposed; inherited scaffold uses it for app/session projections, not Scient project truth | +| Local application state | SQLite for inherited app/session projections; future memory and project-state storage undecided | App SQLite implemented; memory architecture not yet designed | | Cloud database | Postgres | Proposed; not scaffolded | | Cloud platform | Supabase | Initial default candidate; not scaffolded | | Large file storage | Object storage | Proposed; not scaffolded | -| Sync | Local-first SQLite-to-cloud sync | Under evaluation; not scaffolded | +| Sync | Local-first project-state-to-cloud sync | Under evaluation; storage and sync engines not selected or scaffolded | | Application foundation | Standalone Scient-owned, Synara-derived source | Accepted initial foundation through ADR-0001; ownership authority through ADR-0002; scientific product fit remains unproven | | External-agent layer | Synara provider contracts and service | Inherited machinery for external agents; preservation required, project-task compatibility not yet certified | | First-party agent | Scient, derived from standalone Scient-owned, OpenCode-derived source | Accepted identity and source foundation through ADR-0001; ownership authority through ADR-0002; Scient product/runtime not yet implemented | @@ -90,10 +90,10 @@ v0.5.5 passed hosted CI at `d4b10c27` and advanced owned `main` to owned `main` as `d9d8992a`, based on tested upstream `9be46c3c`. Subsequent reviewed UI and project-init status follow-ups advanced maintained `main` to `2ecfbe19`. Standalone ownership and upstream-maintenance follow-ups then -advanced maintained `main` to `d78388a4`; exact provenance is recorded in +advanced maintained `main` to `bd2a6eed`; exact provenance is recorded in `lab/external/sources.lock.md`. The owned OpenCode-derived repository—the current source foundation for the Scient -agent—is in the workspace sibling `../scient-agent/` on `dev` at `14003a01`, +agent—is in the workspace sibling `../scient-agent/` on `dev` at `67e7f3f0`, after a reviewed sync through source version 1.18.3 at official upstream `69a80663` and the standalone upstream-maintenance rollout. Historical Gate 1 and Gate 1.5 commits, tags, and ignored runtime evidence remain @@ -166,9 +166,9 @@ The current lab scaffold has this upstream shape: That tree remains foreign source and should not become the Scient package map by accident. Source-tracing notes and disposable adapter experiments may use -`lab/scient-bridge/`. The first vertical-slice implementation belongs in the -permanent location selected from source evidence during the implementation -plan; do not treat the lab as its default code home. +`lab/scient-bridge/`. The first vertical-slice implementation belongs in a +permanent location to be selected from source evidence and later focused +product/architecture work; do not treat the lab as its default code home. Possible later Scient-owned package areas include: @@ -197,7 +197,11 @@ where practical. Use Electron for the first desktop experiment. The inherited Synara scaffold already provides the Electron shell. -Electron is the pragmatic first choice because Scient needs React, local files, SQLite, subprocesses, agent CLIs, and local background services. These are all easier to integrate in Electron than in a stricter native shell during the first product build. +Electron is the pragmatic first choice because Scient needs React, local files, +embedded structured storage, subprocesses, agent CLIs, and local background +services. These are all easier to integrate in Electron than in a stricter +native shell during the first product build. This shell choice does not select +the canonical project-storage technology. The immediate validation question is whether the Synara-derived shell can host a Scient-owned project mode without forcing scientific work into coding @@ -223,24 +227,32 @@ cloud/project model, not a separate product with separate semantics. ## Local Data -Use SQLite locally. +Use the inherited SQLite boundary for current global app, session, +orchestration, and projection state. Do not infer from that implementation that +SQLite is selected for canonical Scient project records. The inherited scaffold already uses SQLite for Synara app, session, orchestration, and projection state. That database must not be relabeled as the Scient scientific project database. Scient-owned project persistence has not -been designed or implemented. +been selected, designed, or implemented. The future memory-architecture project +will decide the roles of conversations, user memory, project memory, raw +history, files, local application storage, and cloud storage before evaluating +their persistence technologies. Unprocessed questions remain in the +[Idea Inbox](../planning/idea-inbox.md#memory-context-and-continuity). Scient should distinguish: - global app state, such as recent projects, local settings, device identity, and local caches -- per-project scientific state, such as papers, protocol records, evidence records, extraction records, manuscript state, agent runs, and sync metadata +- project-owned scientific state, such as sources, protocol records, evidence records, extraction records, manuscript state, agent runs, and sync metadata -The per-project database is the more important architectural object because projects must be portable and recoverable. +The relationship among project-owned scientific records, future memory, raw +history, and ordinary files remains an open product and architecture question. +Do not turn one candidate representation into architecture before that broader +memory discovery. The inherited scaffold uses Effect SQL with SQLite through Bun. Do not replace -that layer merely to satisfy the target stack before the scaffold baseline is -known. For Scient-owned project persistence, evaluate Effect SQL, Drizzle, -Kysely, or a narrower owned layer after the first project-state contract exists. +that layer merely to satisfy a candidate target stack. Whether any part of a +future memory architecture reuses it is explicitly undecided. ## Cloud Data @@ -270,23 +282,31 @@ Large binary assets should not be stored directly in Postgres. ## Sync -Use local-first sync between local SQLite/project state and the cloud collaboration plane. +The product direction requires local-first sync between project-owned state and +the future cloud collaboration plane. The local representation, cloud authority +model, and exact sync engine are not selected. The exact sync engine is not yet selected. Current candidates: -- PowerSync for SQLite-to-Postgres local-first sync +- PowerSync if a compatible SQLite-to-Postgres topology is proven - Electric for Postgres-backed read sync and live web/cloud views - a Scient-owned sync layer if vendor tools do not fit the required project model -Convex is not selected as the primary database or local-first sync foundation. It may be evaluated for collaboration features or realtime cloud workflows, but the current stack direction requires portable local project state backed by SQLite. +Convex is not selected as the primary database or local-first sync foundation. +It may be evaluated for collaboration features or realtime cloud workflows, but +no candidate may override the requirements for researcher-owned, portable, +recoverable local project state. Scient should maintain domain-level mutation and audit semantics so the product is not locked to one sync vendor. ## Collaboration -Use database sync for structured scientific state. +Use an explicit structured-state synchronization protocol for structured +scientific state once the canonical local representation and cloud authority +model are accepted. Do not assume file-level sync, database sync, or one vendor +before that decision. Use CRDTs only for document-like collaborative surfaces where simultaneous text editing matters. @@ -457,7 +477,7 @@ Completed historical experiments remain evidence, not the roadmap. | Synara-derived application | Standalone owned source, build, isolated Scient identity and state, reviewed upstream process | Scientific-product fit, sustainable domain UI divergence, and long-term maintenance cost | Gate 1 and Gate 1.5 lab reports; ADR-0001 owns adoption; ADR-0002 owns repository authority | | Scient source foundation | Owned OpenCode build, Synara compatibility, project-root fidelity, transcript fidelity, and approval flow for a constrained action | Scient identity and packaging, owned capabilities, isolated Scient state, durable task behavior, and justified inherited-core changes | Gate 1.5 report proves the source baseline; ADR-0001 owns Scient adoption | | External agents | Nine inherited adapters and external OpenCode settings/adapter paths are present in source | Per-agent live compatibility, subscription/auth behavior, project-task certification, and migration protection | [Scient and external agents implementation plan](../planning/scient-and-external-agents-implementation-plan.md) | -| Scient project state | Product responsibilities and trust boundary are documented | Persistence, portable local record, recovery, and first real scientific object relationship | First vertical-slice plan | +| Scient project state and memory | Product responsibilities, high-level memory principles, approved non-Git recovery requirement, and trust boundary are documented | Memory scopes, canonical representation, conversation relationship, package seam, portability, recovery, cloud sync, and first real scientific object relationship | PRD, [Idea Inbox](../planning/idea-inbox.md#memory-context-and-continuity), and future focused architecture work | | Scient-agent and Scient-app boundary | Scient-agent identity plus context, proposal, review, provenance, and permission responsibilities are documented | Actual contract, code placement, event mapping, isolated Scient-agent state, and accepted write-back path | ADR-0001 and linked implementation plans; `agent-runtime.md` remains a future home | | Goose | Source seams, ACP path, and safety risks inspected | Incremental capabilities or architecture lessons for Scient; any future external Goose path is a separate decision | Goose source-depth inspection | | Cloud sync | Postgres, object storage, and local-first sync are proposed directions | Authority, offline behavior, conflicts, revocation, and recovery | Later roadmap and focused architecture work | @@ -474,7 +494,7 @@ The proposed and accepted-by-ADR foundation direction is: TypeScript React Electron -SQLite +SQLite for inherited app state; future memory storage undecided Postgres Supabase as initial cloud platform candidate object storage diff --git a/docs/planning/first-scient-vertical-slice-implementation-plan.md b/docs/planning/first-scient-vertical-slice-implementation-plan.md index e7959eb..e904f7b 100644 --- a/docs/planning/first-scient-vertical-slice-implementation-plan.md +++ b/docs/planning/first-scient-vertical-slice-implementation-plan.md @@ -47,21 +47,30 @@ the next implementation step. ## Current Execution Status Phase 1 is complete. The placement trace selected the permanent package seam, -and desktop PR #4 implemented `@scientfactory/project-init` with zero-write -inspection, explicit plan/apply behavior, conservative recovery, and 43 focused -tests. The package remains present on the maintained desktop `main` at -`2ecfbe19`, after the verified Scient rename on the reviewed official Synara -v0.5.5 foundation and subsequent reviewed UI and project-init status -follow-ups. The owned OpenCode-derived agent-source baseline is -`5ffaf9a2` on `dev`, after the verified `scient-agent` source-boundary rename -and reviewed sync to official upstream `69a80663`. - -Phase 2, the remaining first-slice boundary trace, is the next work. The owned -OpenCode-derived checkout is the Scient agent's selected source foundation, but -the native agent is not yet implemented or packaged. No scientific-state store, -project-initiation UI or server RPC, Scient agent gateway, -proposal/decision ledger, or complete scientific workflow is claimed as -implemented. +and the reviewed desktop implementation added `@scientfactory/project-init` +with zero-write inspection, explicit plan/apply behavior, conservative +recovery, and focused tests. The package, trusted-server RPC, project-open/setup dialog, and +interrupted-initialization recovery UI are present on maintained desktop `main` +at `bd2a6eed6243b13fc1423b21b2454ae060bce5c7`. They initialize only the portable +foundation and do not make the host project projection canonical. + +Phase 2 source tracing is complete in +`../../lab/notes/first-slice-source-trace-2026-07-18.md`. It proved the current +state-ownership gaps and candidate seams. Yaacov clarified that conversations, +project memory, user memory, recovery, cloud synchronization, and their storage +boundaries belong to a dedicated future memory-architecture project, not an +immediate persistence decision. Those unprocessed ideas now live in the +[`Idea Inbox`](idea-inbox.md#memory-context-and-continuity). The permanent +scientific-operation package and fake-executor product proof also remain +deferred. Non-Git recovery, trusted project filesystem scope, and complete +native-Scient/external-OpenCode runtime independence remain product +requirements; they are not a selected memory design. The existing project-init +kernel already enforces path containment for initialization, while later +scientific-operation and native-agent boundaries remain to be built. +The selected OpenCode-derived source is `67e7f3f0341c7a5bad8d68e0a29f113b450eb02a` +on `dev`, but the native agent is not yet implemented or packaged. No +scientific-state store, Scient agent gateway, proposal/decision ledger, or +complete scientific workflow is claimed as implemented. ## Product Slice @@ -396,7 +405,7 @@ Do not design the full project graph or package map. Decide only the first home for project identity, source excerpt, evidence note, task/context, run/proposal/ decision, and recovery responsibilities. -### 3. Produce The Trace Decision +### 3. Produce The Trace And Decision Boundary Create one dated evidence note at: @@ -411,12 +420,14 @@ It must record: 5. the completed state-ownership table; 6. non-Git behavior and recovery findings; 7. the proposed filesystem-scope enforcement boundary; -8. candidate seam comparison and selected permanent placement; +8. candidate seam comparison and either a selected permanent placement or an + explicit owner-approved deferral; 9. existing machinery to reuse unchanged; 10. surfaces Scient must not couple to; 11. any required Synara change, the Scient integration seam, and any proven inherited OpenCode-core gap; -12. the exact first coding backlog; and +12. the exact first coding backlog or the evidence/decision sequence required + before such a backlog is safe; and 13. a go/no-go verdict for implementation. Update this plan only where the trace resolves an open boundary. Update @@ -425,17 +436,27 @@ roadmap or ADR unless the trace invalidates them. ### 4. Review Checkpoint -Stop for Yaacov's review before product code begins. The review should decide -only whether the selected permanent boundary is understandable, serves both -manual and agent work, keeps scientific state outside inherited sessions, -supports credible non-Git recovery, defines enforceable project filesystem -scope, and produces a sufficiently narrow coding backlog. +The first review established three requirements: non-Git recovery, trusted +project filesystem scope, and complete native-Scient/external-OpenCode runtime +independence. Yaacov later clarified that the review had grouped future memory- +architecture questions too narrowly as a current persistence decision. The +scopes and relationships among conversations, user memory, project memory, raw +history, files, local storage, and future cloud storage must be discovered +together in a dedicated future project. -Once accepted, begin implementation. Do not add another exploratory phase -unless the trace identifies a real blocker. +The candidate questions are preserved in the +[`Idea Inbox`](idea-inbox.md#memory-context-and-continuity). They do not select +SQLite, define a project ledger, authorize memory implementation, or block the +independent T3 reliability work. The permanent scientific-operation package and +deterministic fake-executor product proof remain separate deferred decisions. ## Phase 3: Permanent Walking Skeleton +Status: Deferred. The ordered work below remains a candidate sequence, not +current authorization. Replan the memory-dependent steps only when the future +memory-architecture project begins, and revisit the fake-executor question +separately. Do not treat that future discovery as a current T3 dependency. + Implement in this order: 1. **Manual project lifecycle integration.** Connect the reviewed initiation @@ -586,13 +607,13 @@ Stop and report before widening scope if: - controlled fixture; - five-path map; - completed state-ownership table; -- explicit non-Git recovery approach; -- selected permanent code location; +- approved non-Git recovery requirement and unresolved future memory handoff; +- permanent code-location candidates and explicit owner deferral; - reuse-versus-change list; - Scient integration and inherited OpenCode-core change verdict; - Synara-hosted versus external-topology verdict; -- narrow first coding backlog; and -- Yaacov's approval to implement. +- future memory questions routed to the idea inbox; and +- no implied authorization for scientific-state or memory implementation. ### Vertical Slice Done diff --git a/docs/planning/idea-inbox.md b/docs/planning/idea-inbox.md index 2e9aa02..579df53 100644 --- a/docs/planning/idea-inbox.md +++ b/docs/planning/idea-inbox.md @@ -22,9 +22,11 @@ remove the raw inbox entry. Do not leave duplicate copies here and elsewhere. ## Entry Shape -Use the lightest useful form: +Group entries under the nearest broad area and use the lightest useful form: ```md +### Area + | Idea | Raised by | Date added | Context or source | Possible area | |---|---|---|---|---| | **Short title.** Description. | Person's name | YYYY-MM-DD | ... | Product, architecture, design, research, quality, or unknown | @@ -35,6 +37,10 @@ is being triaged. `Raised by` identifies the human source of the idea, not the person or agent who edited this file. If the source cannot be established, write `Not recorded` rather than guessing. +An unusually broad idea may have a short question inventory below its table row +when that is necessary to preserve the raw scope. The inventory remains intake, +not analysis or architecture, and should move with the idea when it is promoted. + ## Triage Destinations - Product candidates and open product questions: `product-planning.md` @@ -46,7 +52,96 @@ write `Not recorded` rather than guessing. ## Unprocessed Ideas +### Research Exploration And Visualization + | Idea | Raised by | Date added | Context or source | Possible area | |---|---|---|---|---| | **Visual literature map.** Add an interactive, Obsidian-style graph view of the literature sources in a Scient project. Sources would appear as nodes, with inspectable relationships such as citations, shared topics, project links, or researcher-created connections. The view should help researchers explore clusters, identify central or isolated sources, review gaps, filter the collection, and open each source in its normal detail view. | Yishai | 2026-07-18 | Spoken idea. The initial scope should focus on sources already imported into the project; broader scholarly-network discovery and the exact relationship types remain open questions. | Source-library product planning, literature-review UX, design, and future source-relationship architecture | -| **Future Scient memory architecture.** Plan Scient's complete memory architecture as a dedicated future product and architecture discovery before selecting schemas, databases, or synchronization machinery. | Not recorded | 2026-07-18 | Questions about project records, conversations, candidate memory scopes, provenance, SQLite, Git, user-selected cloud folders, recovery, privacy, retention, and future Scient cloud sync arose during the first-slice T3 source review. The scopes and boundaries remain unaccepted, and no storage choice is authorized. Detailed candidate scopes and open questions are preserved in [draft PR #23](https://github.com/ScientFactory/Scient/pull/23). | Product planning, future memory architecture, agent runtime, project format, security, provenance, synchronization, and product design | + +### Memory, Context, And Continuity + +| Idea | Raised by | Date added | Context or source | Possible area | +|---|---|---|---|---| +| **Future Scient memory architecture.** Discuss Scient's complete memory architecture as a dedicated future product and architecture project before selecting schemas, databases, or synchronization machinery. | Yaacov | 2026-07-18 | Questions about project records, conversations, SQLite, Git, user-selected cloud folders, recovery, and future Scient cloud sync arose during the first-slice source review. They belong to the broader memory-system discussion, not the completed T3 reliability work or an immediate persistence decision. Reusable questions from an oversized standalone persistence brief were condensed here before that out-of-scope architecture file was removed. | Product planning, future memory architecture, agent runtime, project format, security, provenance, synchronization, and product design | + +#### Candidate scopes and questions to preserve + +Candidate scopes to discuss, not accepted layers: + +- **Conversation memory:** short-lived or derived context from one conversation, + distinct from the complete transcript. +- **Task or run memory:** working state for one delegated task or agent run, + including what may expire when the run completes. +- **Project memory:** durable project direction, decisions, source judgments, + analysis choices, unresolved questions, collaborator decisions, and prior + work needed for continuity. +- **User memory:** personal preferences, recurring choices, and working style + that may apply across projects without silently overriding project rules. +- **Team or organization memory:** shared methods, conventions, and + institutional knowledge with explicit membership, permission, and ownership + boundaries. +- **Scient-maintained knowledge:** built-in product guidance, scientific skills, + and maintained procedures; this may require different authority and update + rules from user-generated memory. +- **Raw history and provenance:** complete conversations, events, actions, and + evidence that may support memory but are not automatically trusted memory. + +Questions for the future discovery project: + +- Which candidate scopes are actually needed, and what are their precise names + and responsibilities? +- Who owns, reads, edits, shares, exports, deletes, or promotes information in + each scope? +- What is ephemeral, retained for continuity, durable, canonical, derived, or + rebuildable? +- What is the difference between a complete conversation transcript, + conversation context, a summary, and trusted memory? +- Can a conversation or task propose project memory, and which promotions + require explicit researcher review? +- How are source, authority, confidence, freshness, conflict, staleness, + distrust, archival, forgetting, and supersession represented? +- What happens when user memory conflicts with project memory, or project + memory conflicts with current files and evidence? +- How can researchers inspect, correct, pin, challenge, archive, forget, or + disable remembered information? +- Which memory can Scient use, and which memory may be disclosed to an external + agent for one bounded task? +- How does task context include only the appropriate user, project, + conversation, and run information? +- How do project memory and provenance relate to ordinary researcher-owned + files without turning generated summaries into project authority? +- How do projects remain friendly to Git while never requiring Git for ordinary + use, history, or recovery? +- How should projects behave in iCloud Drive, Dropbox, OneDrive, external + drives, network locations, and other user-selected folders? +- Which information belongs in the project folder, local application storage, + a user account, or future Scient cloud storage? +- What remains fully useful offline, and what is restored or synchronized when + cloud access returns? +- How are concurrent edits, offline divergence, deletion, restoration, device + loss, and collaborator removal handled? +- How are conversations, memory, decisions, and provenance retained, backed up, + exported, transferred, encrypted, redacted, or permanently deleted? +- Which privacy, sensitive-data, institutional-control, and regional-storage + requirements constrain the design? +- What scale, latency, reliability, recovery, migration, portability, and exit + requirements must be accepted before evaluating storage technologies? +- Only after the memory model is understood, which persistence approaches + should be evaluated, including SQLite, structured files, derived indexes, + local databases, cloud databases, and combinations of them? +- If SQLite remains a candidate at that later stage, what evidence is required + for crash behavior, journal/WAL handling, active copying, cloud-folder sync, + backup and restore, integrity checks, locking, migrations, packaging, + performance, readable export, and exit from the format? + +Explicit non-decisions: + +- These scopes are prompts for discussion, not accepted memory architecture. +- Conversation, user, project, task, team, and system memory are not yet formal + product objects or storage boundaries. +- No database, schema, project ledger, cloud-sync protocol, or retention policy + is selected or authorized. +- The existing application SQLite database remains current app/session + implementation; it does not settle the future memory architecture. +- This future discovery does not block T3 reliability work or unrelated product + work. diff --git a/docs/planning/scient-and-external-agents-implementation-plan.md b/docs/planning/scient-and-external-agents-implementation-plan.md index e9baff6..26fb4b1 100644 --- a/docs/planning/scient-and-external-agents-implementation-plan.md +++ b/docs/planning/scient-and-external-agents-implementation-plan.md @@ -125,7 +125,7 @@ Source lineage does not merge product identities: ## Current Implementation Truth At maintained desktop-source revision -`d78388a42bcc09dabc926c0885ec34a8de6427b0`, inspected on 2026-07-18, the +`bd2a6eed6243b13fc1423b21b2454ae060bce5c7`, inspected on 2026-07-18, the inherited host contains a shared provider adapter contract and adapters for: - Codex; @@ -158,7 +158,7 @@ paths first, then certify compatibility honestly per agent. The owned OpenCode-derived checkout in the workspace sibling `../scient-agent/` (relative to the Scient repository root), maintained on -`dev` at `5ffaf9a2dfa5b958e8f4856b94b50d26b00c6b76`, is the accepted +`dev` at `67e7f3f0341c7a5bad8d68e0a29f113b450eb02a`, is the accepted source foundation for the Scient agent. Its repository and maintenance-verifier identity are Scient-owned while its current runtime remains upstream-aligned OpenCode source. Scient-agent packaging, private state diff --git a/docs/research/source-evaluations/open-source-adaptation-map.md b/docs/research/source-evaluations/open-source-adaptation-map.md index 88ce9aa..3272f82 100644 --- a/docs/research/source-evaluations/open-source-adaptation-map.md +++ b/docs/research/source-evaluations/open-source-adaptation-map.md @@ -57,6 +57,10 @@ Current inputs: [Lacuna: A Research Map for Machine Learning](https://arxiv.org/html/2606.26246v1) and live site as a research-map reference for literature search, synthesis, and agent-readable paper-grounded intermediate objects. +- Targeted T3 Code inspection through revision + `bf76535fe4da71d8de7b8bd5ffa0d2086b7af8d0` on 2026-07-18, with + accepted, deferred, and rejected dispositions recorded in + [`t3-code-targeted-review-2026-07-18.md`](../../../lab/notes/t3-code-targeted-review-2026-07-18.md). Remaining evidence gaps before architecture promotion: @@ -330,13 +334,16 @@ is relevant only to a separately reviewed process or external-agent path. | Source | Adaptation target | Why it matters | Do not adopt | Depth status | |---|---|---|---|---| -| T3 Code | Desktop/backend process lifecycle, provider-instance patterns, remote/SSH/Tailscale ideas if needed, multi-surface product structure. | Gives practical patterns for a desktop agent app that coordinates backends and providers. | Do not inherit coding-product assumptions. | Needs targeted review of provider-instance and process lifecycle code. | +| T3 Code | Bounded reliability fixes now; provider-instance separation as design evidence for execution targets above `ProviderKind`; lifecycle and diagnostics patterns only when a concrete trigger appears. | Supplies proven implementation details and comparison evidence without becoming Scient's product architecture. | Do not inherit coding-product assumptions, broad runtime alignment, mobile/cloud surfaces, or speculative remote infrastructure. | Targeted review completed through `bf76535fe4da71d8de7b8bd5ffa0d2086b7af8d0` on 2026-07-18. Three reliability fixes were accepted, snapshot startup was characterized, browser socket isolation was proven and fixed, and all larger ideas were explicitly deferred or rejected. | | Synara | Orchestration, UI/provider adapters, Effect server ideas, event-sourced orchestration, desktop/web split, worktree/Git flows. | Useful for building a reliable agent workspace that can explain what happened. | Do not copy its UI shape blindly; Scient needs a research cockpit. | Accepted initial application foundation through ADR-0001; scientific-product fit still needs pressure testing. | | Vercel AI SDK | Model/provider abstraction, typed stream parts, tool-call state, approval status, UI message events, mock providers, and model I/O tests. | Useful for model plumbing and chat/event surfaces around Scient-owned actions. | Do not use it as the abstraction over local executors like OpenCode or Codex. Executor actions need a Scient-owned contract. | Candidate model I/O layer; needs a narrow harness prototype. | | Vercel AI Elements | Tool cards, source citations, confirmations, terminal output, file trees, artifacts, plans, queue state. | Useful UI pieces for agent work inspection. | Do not let it make Scient a generic chat surface. | Side UI pattern source. | -Recommendation: adapt shell/process/provider ideas from T3 Code and Synara, but -keep the scientific navigation and object model Scient-owned. +Recommendation: keep Synara as the owned application foundation and use T3 Code +only as a trigger-driven donor. The completed T3 review does not create an +ongoing upstream-monitoring obligation. Provider-instance patterns may inform +the already-planned execution-target contract, while scientific navigation, +canonical state, provenance, review, and recovery remain Scient-owned. ### Desktop Base And Science-App Candidates diff --git a/lab/external/owned-sources.json b/lab/external/owned-sources.json index 05e5ae9..83cdddf 100644 --- a/lab/external/owned-sources.json +++ b/lab/external/owned-sources.json @@ -5,7 +5,7 @@ "ownedRepository": "ScientFactory/scient-desktop", "sourceLockLabel": "Scient desktop", "ownedDefaultBranch": "main", - "testedHead": "57e6b2cde09f64db367b894506f56db605fb91b4", + "testedHead": "bd2a6eed6243b13fc1423b21b2454ae060bce5c7", "officialRepository": "Emanuele-web04/synara", "officialDefaultBranch": "main", "reviewedThrough": "69304bc1d59d86da8afbac367118c75db8c9dbfe", @@ -18,7 +18,7 @@ "ownedRepository": "ScientFactory/scient-agent", "sourceLockLabel": "Scient agent source", "ownedDefaultBranch": "dev", - "testedHead": "bc125cbc60c36e4b7013f8d7cf755f745af509b3", + "testedHead": "67e7f3f0341c7a5bad8d68e0a29f113b450eb02a", "officialRepository": "anomalyco/opencode", "officialDefaultBranch": "dev", "reviewedThrough": "fab213312927ea64cf968832c527206e8c944f9e", diff --git a/lab/external/sources.lock.md b/lab/external/sources.lock.md index ce62d35..62fedde 100644 --- a/lab/external/sources.lock.md +++ b/lab/external/sources.lock.md @@ -26,10 +26,10 @@ siblings. A deferred source with no retained checkout says so explicitly. | Source | Local path | Official upstream | Owned repository | Tested integrated upstream base | Maintained/tested commit | Role and update mode | |---|---|---|---|---|---|---| -| Scient agent source (OpenCode-derived) | `../scient-agent/`; canonical workspace sibling on `dev` at `bc125cbc60c36e4b7013f8d7cf755f745af509b3` | `https://github.com/anomalyco/opencode.git`, `dev` | `https://github.com/ScientFactory/scient-agent`, public standalone repository | `69a80663a2ed7d671d2b4d5dd6f2d605714675a5` | Current owned `dev` `bc125cbc60c36e4b7013f8d7cf755f745af509b3`; exact rename and maintenance evidence below | Owned source foundation for the planned Scient agent; `adapter-maintained`; native Scient runtime identity is not yet implemented. | +| Scient agent source (OpenCode-derived) | `../scient-agent/`; canonical workspace sibling on `dev` at `67e7f3f0341c7a5bad8d68e0a29f113b450eb02a` | `https://github.com/anomalyco/opencode.git`, `dev` | `https://github.com/ScientFactory/scient-agent`, public standalone repository | `69a80663a2ed7d671d2b4d5dd6f2d605714675a5` | Current owned `dev` `67e7f3f0341c7a5bad8d68e0a29f113b450eb02a`; exact rename and maintenance evidence below | Owned source foundation for the planned Scient agent; `adapter-maintained`; native Scient runtime identity is not yet implemented. | | Goose | No local checkout is retained. | `https://github.com/aaif-goose/goose.git`, `main` | None; owned repository deferred | Not tested in Gate 1.5 | Last inspected commit `3c1fdd692cc8aaa5f09b9175410c09a09d4dfe49` | Deferred broader-agent research input. Repository, build, ACP adapter, runtime, credentials, and adoption wait until after the first Scient gateway. | -| Scient desktop (Synara-derived) | `../scient-desktop/`; canonical workspace sibling on `main` at `57e6b2cde09f64db367b894506f56db605fb91b4` | `https://github.com/Emanuele-web04/synara.git`, `main` | `https://github.com/ScientFactory/scient-desktop`, public standalone repository | `9be46c3ce6a7521b64436b7334bc6fce16e3cac4` | Current owned `main` `57e6b2cde09f64db367b894506f56db605fb91b4`; exact rename and maintenance evidence below | Accepted initial application foundation; `divergent-cherry-pick`; must not own scientific project truth. | -| T3 Code | No local checkout is retained. | `https://github.com/pingdotgg/t3code.git`, `main` | None | Not tested in Gate 1.5 | Last inspected commit `b9cc8d6ef17ca9f45bec621bef71ad3f706b9276` | Desktop/runtime/provider/process reference only. | +| Scient desktop (Synara-derived) | `../scient-desktop/`; canonical workspace sibling on `main` at `bd2a6eed6243b13fc1423b21b2454ae060bce5c7` | `https://github.com/Emanuele-web04/synara.git`, `main` | `https://github.com/ScientFactory/scient-desktop`, public standalone repository | `9be46c3ce6a7521b64436b7334bc6fce16e3cac4` | Current owned `main` `bd2a6eed6243b13fc1423b21b2454ae060bce5c7`; exact rename and maintenance evidence below | Accepted initial application foundation; `divergent-cherry-pick`; must not own scientific project truth. | +| T3 Code | No canonical local checkout is retained. | `https://github.com/pingdotgg/t3code.git`, `main` | None | Not tested in Gate 1.5 | Targeted review completed through `bf76535fe4da71d8de7b8bd5ffa0d2086b7af8d0`; see [`t3-code-targeted-review-2026-07-18.md`](../notes/t3-code-targeted-review-2026-07-18.md) | Trigger-driven desktop/runtime/provider/process reference only; not a continuously monitored upstream. | ## Maintained Upstream Review State @@ -40,8 +40,8 @@ review is accepted. | Source | Tested owned head | Reviewed through | Integration base | Update mode | Review evidence | |---|---|---|---|---|---| -| Scient desktop | `57e6b2cde09f64db367b894506f56db605fb91b4` | `69304bc1d59d86da8afbac367118c75db8c9dbfe` on 2026-07-18 | `9be46c3ce6a7521b64436b7334bc6fce16e3cac4` | `divergent-cherry-pick` | [`2026-07-18-scient-desktop.md`](upstream-reviews/2026-07-18-scient-desktop.md); no code intake | -| Scient agent source | `bc125cbc60c36e4b7013f8d7cf755f745af509b3` | `fab213312927ea64cf968832c527206e8c944f9e` on 2026-07-18 | `69a80663a2ed7d671d2b4d5dd6f2d605714675a5` | `adapter-maintained` | [`2026-07-18-scient-agent.md`](upstream-reviews/2026-07-18-scient-agent.md); no code intake | +| Scient desktop | `bd2a6eed6243b13fc1423b21b2454ae060bce5c7` | `69304bc1d59d86da8afbac367118c75db8c9dbfe` on 2026-07-18 | `9be46c3ce6a7521b64436b7334bc6fce16e3cac4` | `divergent-cherry-pick` | [`2026-07-18-scient-desktop.md`](upstream-reviews/2026-07-18-scient-desktop.md); no code intake | +| Scient agent source | `67e7f3f0341c7a5bad8d68e0a29f113b450eb02a` | `fab213312927ea64cf968832c527206e8c944f9e` on 2026-07-18 | `69a80663a2ed7d671d2b4d5dd6f2d605714675a5` | `adapter-maintained` | [`2026-07-18-scient-agent.md`](upstream-reviews/2026-07-18-scient-agent.md); no code intake | ## Standalone Ownership And Maintenance Rollout @@ -86,6 +86,12 @@ source-repository pull requests: distribution-signed and fails strict code-sign verification, so public release remains blocked by [issue #6](https://github.com/ScientFactory/scient-desktop/issues/6). +- Desktop [PR #14](https://github.com/ScientFactory/scient-desktop/pull/14) + adapted six bounded T3-informed reliability protections and browser-session + isolation fixes; exact head `8a8398e817b803ccf7811d6f4f378bee1fa85d77` + passed hosted CI run `29646625880`, including browser, Windows process, build, + and release-smoke coverage, and merged as + `bd2a6eed6243b13fc1423b21b2454ae060bce5c7`. - Agent [PR #1](https://github.com/ScientFactory/scient-agent/pull/1) established the operator card, review state, verifier modes, owned source quality workflow, and monitor; exact head @@ -123,6 +129,12 @@ source-repository pull requests: `e6a06de63ce668144bb00e20dbd8f392a4da230a` passed hosted source-quality run `29645764713` without annotations and merged as `bc125cbc60c36e4b7013f8d7cf755f745af509b3`. +- Agent [PR #8](https://github.com/ScientFactory/scient-agent/pull/8) + refreshed the immutable `apple-actions/import-codesign-certs` pin in the + retained, upstream-guarded publish workflow; exact head + `08a9ccec50bf1100d9ad32211a9bbaec0db06717` passed hosted source-quality run + `29646968673` and merged as + `67e7f3f0341c7a5bad8d68e0a29f113b450eb02a`. No source code from the reviewed official ranges was integrated during this rollout. The PRs above establish ownership, review, monitoring, and verification diff --git a/lab/notes/README.md b/lab/notes/README.md index 050d44c..058db2b 100644 --- a/lab/notes/README.md +++ b/lab/notes/README.md @@ -20,6 +20,8 @@ integration observations, and temporary lab decisions. - `synara-gate-1-baseline-2026-07-11.md` - historical inherited-scaffold baseline, official-CLI correction run, and Gate 1 pass result. - `synara-first-inspection-2026-07-07.md` - first source inspection of Synara as desktop base, OpenCode first-agent path, Goose integration path, and Scient ownership boundary. - `goose-source-depth-inspection-2026-07-11.md` - research input for Goose's later role, ACP surfaces, runtime-state boundary, permission risks, and first adapter recommendation. +- `first-slice-source-trace-2026-07-18.md` - exact desktop/agent path trace for the first scientific source-to-note slice; future memory architecture and the permanent package remain explicitly unselected. +- `t3-code-targeted-review-2026-07-18.md` - bounded T3 Code review through `bf76535fe4da71d8de7b8bd5ffa0d2086b7af8d0`, including accepted reliability work and explicit deferred/rejected dispositions. Keep notes clear about whether they are: diff --git a/lab/notes/first-slice-source-trace-2026-07-18.md b/lab/notes/first-slice-source-trace-2026-07-18.md new file mode 100644 index 0000000..276d659 --- /dev/null +++ b/lab/notes/first-slice-source-trace-2026-07-18.md @@ -0,0 +1,378 @@ +# First Scientific Slice Source Trace + +Status: Complete as source evidence; memory and package architecture not selected +Owner: Yaacov +Created: 2026-07-18 +Last updated: 2026-07-18 +Purpose: Maps the first scientific slice's current source seams and proven gaps without selecting memory layers, persistence technology, or permanent product architecture. +Doc type: Implementation evidence + +## Verdict + +The maintained Synara-derived desktop can host a later scientific workflow +without making its project, thread, provider, or checkpoint projections +canonical scientific memory. This source trace does not select the future +memory architecture, its storage, or the permanent scientific-operation package. + +A hybrid placement was one source-trace candidate, not an accepted design: + +1. a possible `packages/scient-project` package in `scient-desktop` owns the + first-slice domain rules, proposal decisions, recovery behavior, and + filesystem-scope policy; +2. thin contracts, trusted-server RPC, and project-level UI shims call that + package; and +3. a narrow executor port accepts immutable task/context receipts and returns a + proposal. Deterministic fake-executor implementation is deferred for later + discussion. Scient would eventually implement the port as a distinct + execution target; external OpenCode remains an independent adapter. + +The trace proved that `.scient/project.json` already owns portable project +identity while ongoing scientific memory and records have no selected location +or representation. Conversations, task/run context, project memory, user +memory, raw history, files, recovery, and cloud sync must be discussed together +in a dedicated future memory-architecture project. The raw candidate layers and +questions are preserved in the +[`Idea Inbox`](../../docs/planning/idea-inbox.md#memory-context-and-continuity). + +The existing Git-backed checkpoint mechanism is not suitable for the approved +non-Git guarantee. A future memory and recovery design must determine how +accepted work remains coherent and recoverable without making a Git repository, +worktree, executor session, or transcript the only reconstruction path. The +mechanism and record model are open. + +One required and user-approved native-agent gap is now proven: the selected +`scient-agent` source still defaults its global data/config/state roots and +database name to `opencode`. Native Scient packaging must supply a dedicated +first-party runtime profile before it can be connected. That change must not +alter the existing external OpenCode adapter or its user-owned data paths. + +## Controlled Fixture + +The deterministic synthetic capsule below remains a candidate for a later +controlled memory/recovery or executor test. Yaacov deferred fake-executor product +implementation and is separately preparing a real project for later manual +testing. The controlled fixture would protect the real project from destructive +crash, corruption, migration, and recovery experiments; it is not authorized by +this trace. + +The capsule is `greenhouse-seedling-capsule` and lives outside every parent Git +repository during runtime tests. Its committed test fixture will contain: + +- `README.md`: project title, question, and short plan; +- `sources/trial-summary.txt`: a locally authored source summary with one + designated exact excerpt; +- `data/seedling-heights.csv`: a small deterministic dataset; +- `analysis/summary.txt`: precomputed group means and the command used to + reproduce them; +- `outputs/height-by-treatment.svg`: one representative figure; +- `writing/results-fragment.md`: a short results fragment; +- `expected/stages.json`: deterministic initialized, source-added, + task-created, proposed, accepted, reopened, and recovered states; and +- no network, cloud service, parser, sensitive data, or external API. + +The first exercised thread is deliberately narrower than the capsule: manually +select one exact excerpt, create the task “Summarize this excerpt as a +two-sentence evidence note using only the selected text,” produce one +deterministic proposed note, inspect its source link, accept or reject it, +reopen the project, and prove recovery. The data, analysis, figure, and writing +files are context for future slices and are not ingested in this one. + +## Selected Sources And Health + +| Source | Selected revision | Working state | Relevant upstream result | Minimal health evidence | +|---|---|---|---|---| +| Scient desktop (`scient-desktop`) | owned `main` `bd2a6eed6243b13fc1423b21b2454ae060bce5c7` | Canonical checkout clean when selected | Official Synara reviewed through `69304bc1d59d86da8afbac367118c75db8c9dbfe`; verifier current with no unreviewed commits | Existing installed-app smoke reached project initialization. Reliability PR #14 passed full tests, typecheck, build, release smoke, and 171 browser tests at exact head `8a8398e8` before merging; the change does not alter this trace's product seams. | +| Scient agent (`scient-agent`) | owned `dev` `67e7f3f0341c7a5bad8d68e0a29f113b450eb02a` | Clean and equal to `origin/dev` | Official OpenCode reviewed through `fab213312927ea64cf968832c527206e8c944f9e`; verifier current with no unreviewed commits | Maintained upstream verifier and public-identity check passed. A live Scient action was not run because native Scient identity, private state, packaging, and gateway are correctly still absent. | + +No upstream code was merged during this trace. The agent revision advanced only +through already-merged repository automation changes after the prior source-lock +snapshot; the runtime files inspected below remain selected source, not proof of +a native Scient product. + +## Path A: Project Creation, Opening, And Reopening + +### Current path + +| Stage | Exact entry point | State and behavior | +|---|---|---| +| Folder selection and setup choice | `apps/web/src/components/Sidebar.tsx`: `addProjectFromPath`, `requestProjectInitializationDecision`; `apps/web/src/components/ScientProjectInitializationDialog.tsx` | Opening first calls the Scient initialization preview. The user may initialize, recover, roll back, open without setup, or cancel. Opening without setup makes no Scient initialization write. | +| Shared initialization flow | `apps/web/src/lib/scientProjectInitialization.ts`: `prepareScientProjectForOpening` | Re-previews after failures, consumes one-time preview capabilities, reports completion, and keeps initialization separate from project registration. | +| Trusted filesystem application | `apps/server/src/scientProjectInitialization.ts`: `ScientProjectInitializationService`; `packages/scient-project-init` | The server calls the existing dependency-light kernel. Preview capabilities expire and are single-use. Kernel preconditions, path containment, transaction recovery, and preserve/propose/conflict behavior remain authoritative. | +| Host project registration | `apps/web/src/lib/projectCreation.ts`: `createOrRecoverProjectFromPath`; `apps/server/src/orchestration/decider.ts`: `project.create`; `apps/server/src/orchestration/projectMetadataProjection.ts`: `applyProjectMetadataProjection` | A Synara project ID, title, workspace root, default model, and timestamps are persisted as events and `projection_projects`. Duplicate roots are recovered or rejected. This registration can survive restart but is a host index, not Scient project identity. | +| Restart read model | `apps/server/src/orchestration/Layers/ProjectionPipeline.ts`; `apps/server/src/orchestration/Layers/ProjectionSnapshotQuery.ts`; `apps/web/src/store.ts` | SQLite projections rebuild the shell snapshot and sidebar. Failure to synchronize is surfaced as a retryable add-project error. | + +Desktop production state derives from `SCIENT_HOME` (normally `~/.scient`). +`apps/server/src/config.ts:deriveServerPaths` places host events, projections, +settings, and provider session projections in +`~/.scient/userdata/state.sqlite`. Electron/Chromium UI state uses the separate +platform user-data profile resolved by +`apps/desktop/src/desktopUserDataProfile.ts` (on macOS, +`~/Library/Application Support/scient`). + +### Scient attachment point + +After the host resolves a workspace root, later project-aware capabilities can +inspect `.scient/project.json` and key their UI to the portable Scient project +ID while keeping the Synara project ID as a replaceable host reference. This +trace does not decide which memory or record boundary is then opened, what must +move with the folder, or how export, restore, and cloud continuation work. +Existing host or chat state must not be described as accepted memory merely +because no future store has been selected. + +## Path B: Manual Operations And Minimal UI + +The current product has the necessary shell but no canonical scientific +operations. A candidate UI placement is a new project-level route and component, +not another chat-thread projection: + +- add `_chat.project.$projectId.tsx` as the thin route shim; +- add a focused `ScientProjectView` with source, task/context, proposal, and + history sections; +- enter it from the existing project row/context menu in `Sidebar.tsx`; +- reuse existing dialog, button, form, toast, loading, and error primitives; +- reuse `WorkspaceFilePreview` only for optional local-file inspection; and +- use a small structured before/proposed-note review card. Do not force the + note through Git/file diff machinery merely to reuse `DiffPanel`. + +Both UI actions and later executor output must call the same Scient-owned domain +commands through a trusted boundary. Whether that boundary is a package or +another reviewed seam remains open. + +- add or revise a source excerpt; +- create a task and immutable context receipt; +- begin/finish/fail a run receipt; +- record a proposal linked to exact source and context revisions; +- edit the proposal without accepting it; +- accept or reject with a decision record; and +- create or apply a recovery action. + +The existing `EditorWorkspaceView`, chat composer, Kanban board, and +thread-proposed-plan surfaces may remain available, but none owns source, +evidence, task, proposal, or decision truth for this slice. + +## Path C: Scient Execution And External OpenCode Separation + +### Reusable inherited path + +| Concern | Exact current seam | Reuse decision | +|---|---|---| +| Provider contract | `apps/server/src/provider/Services/ProviderAdapter.ts`: `ProviderAdapterShape` | Reuse provider session, send, interrupt, request response, thread read, and canonical event concepts behind a new executor port. Do not expose `ProviderKind` as Scient's durable execution-target identity. | +| Session orchestration | `apps/server/src/orchestration/Layers/ProviderCommandReactor.ts`: session start/restart, cwd resolution, prompt assembly, interrupt, approval dispatch | Reuse as external-agent infrastructure. Native Scient may later adapt to the same normalized lifecycle, but scientific context must be assembled by the Scient gateway from an immutable context receipt. | +| External OpenCode | `apps/server/src/provider/Layers/OpenCodeAdapter.ts`: `startSession`, `sendTurn`, `interruptTurn`, `respondToRequest`; `apps/server/src/provider/opencodeRuntime.ts`: `OPENCODE_CLI_SPEC` | Preserve unchanged as the user's independently installed/connected OpenCode. It keeps its own binary/URL/password, `opencode` data directory, credentials, sessions, and updates. | +| Runtime evidence | `packages/contracts/src/providerRuntime.ts`: `ProviderRuntimeEvent` family; `apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts` | Reuse normalized session, turn, item, content, approval, tool, file, warning, error, and completion events as execution evidence. They do not become canonical scientific records. | +| Native source | owned `scient-agent`: `packages/core/src/global.ts`; `packages/core/src/database/database.ts` | Selected inherited core, but not yet a native product runtime. It currently hardcodes the application path segment `opencode` and default database names `opencode*.db`; this is the exact state-isolation gap to close. | + +### Execution contract + +A later Scient-owned `ScientificTaskExecutor` candidate would accept only: + +- stable execution-target ID; +- Scient project ID and run ID; +- project-root capability reference, not an arbitrary model-generated path; +- immutable task revision; +- immutable context receipt with selected source revision IDs and hashes; +- allowed operation/capability set; and +- cancellation signal. + +Its output would be a proposed record plus normalized run evidence. It cannot +accept scientific state and cannot write canonical records directly. The +trusted Scient boundary would record the run and proposal only after validating +IDs, hashes, scope, and current revisions. + +### Filesystem scope + +`apps/server/src/workspace/Layers/WorkspacePaths.ts` and +`WorkspaceFileSystem.ts` already provide useful lexical and real-path +containment for app file RPC. They should be reused by server shims, including +symlink checks, but they do not confine provider-native tools or a shell. + +The selected agent source improves mutation safety in +`packages/core/src/location-mutation.ts`, which rejects relative escapes and +requires `external_directory` permission for explicit external mutation paths. +However, `packages/core/src/tool/bash.ts` explicitly states that command-argument +path scanning is advisory and that the shell runs with host-user filesystem, +process, and network authority. A cwd plus approval mode is therefore not an +enforceable project sandbox. + +Approved scope requirements for a later slice are: + +1. canonical operations accept only package IDs and root-relative paths; +2. any controlled executor used for proof has no filesystem capability unless + a separately reviewed capability is required; +3. native Scient later runs with a dedicated capability profile that disables + unrestricted shell/direct-write tools for this workflow and exposes only + scoped Scient gateway tools; +4. every gateway path is resolved against the registered real project root and + rechecked after following existing ancestors/symlinks; and +5. proposal acceptance, not model output, is the only route to canonical state. + +If a later workflow genuinely requires arbitrary shell execution, add an OS +sandbox or isolated staged workspace as a separate safety slice. Do not claim +that the current OpenCode permission prompt alone enforces filesystem scope. + +## Path D: Current State Ownership And Future Memory Questions + +| State | Current owner | Location | Survives restart? | Canonical for Scient? | +|---|---|---|---|---| +| Application identity and settings | Synara-derived Scient host | Electron profile under platform app data; server settings at `~/.scient/userdata/settings.json` | Yes | No | +| Workspace path | Synara-derived host | `projection_projects.workspace_root` in `~/.scient/userdata/state.sqlite` and UI shell state | Yes | Host reference only | +| Synara project/session projection | Synara-derived host | orchestration events and projection tables in `~/.scient/userdata/state.sqlite` | Yes | No | +| Scient session and transcript | Missing; OpenCode-derived source selected | Must use a dedicated first-party agent data/config/state root and database, separate from external OpenCode | Must for runtime continuity | No | +| External OpenCode session and transcript | External OpenCode plus host adapter | External OpenCode's XDG/Application Support `opencode` data; resume/session projection in host SQLite | Provider-dependent; normally yes | No | +| Runtime events and tool logs | Agent plus host ingestion | Provider-native store; normalized host events/projections; optional `~/.scient/userdata/logs/provider/events.log` | Yes where persisted | Evidence only | +| Scient project identity | Existing project-init kernel | `.scient/project.json` | Yes and moves with folder | Yes | +| Source excerpt and evidence note | Missing | Project-owned representation and location undecided | Must | Must | +| Task and context receipt | Missing | Project-owned representation and location undecided | Must | Must | +| Run receipt, proposal, and decision | Missing | Project-owned representation and location undecided | Must | Must | +| Recovery state | Missing | Project-owned representation and location undecided | Must | Must | + +The future memory-architecture project must decide which of these rows represent +memory, raw history, provenance, ordinary files, application state, or another +kind of project record before comparing storage technologies. Its questions +include portability, Git independence and compatibility, user-selected cloud +folders, recovery, conversation retention, future Scient cloud sync, privacy, +and export. They are preserved in the +[`Idea Inbox`](../../docs/planning/idea-inbox.md#memory-context-and-continuity). +Secrets and provider credentials must never become project memory merely for +convenience. + +## Path E: Proposal, Review, Non-Git Recovery, And Reopening + +Current checkpointing is intentionally Git-specific: + +- `apps/server/src/checkpointing/Services/CheckpointStore.ts` defines hidden + Git-ref capture, restore, reverse diff, and checkpoint diff; +- `apps/server/src/checkpointing/Layers/CheckpointStore.ts` implements those + operations with an isolated Git index and refs; and +- `apps/server/src/orchestration/Layers/CheckpointReactor.ts` emits + “Checkpoints are unavailable because this project is not a git repository” + for the non-Git boundary. + +Provider live diffs and host proposed plans are useful presentation/evidence +inputs, but they remain thread/provider projections and cannot reconstruct an +accepted scientific note after their session is deleted. + +The trace sketched the following candidate later proposal flow. Only the high- +level non-Git recovery outcome is approved; these record and transaction details +are not accepted memory architecture: + +1. create immutable source, task, and context revisions; +2. open a run receipt before executor invocation; +3. record executor evidence and a proposal linked to exact revision IDs; +4. allow proposal text edits as new proposal revisions; +5. on accept, begin one atomic or equivalently recoverable publish operation; +6. append or preserve a recovery point containing the prior accepted state; +7. append the decision and accepted note revision; +8. atomically advance the accepted state and publish; and +9. on reject or failure, append the outcome without changing accepted pointers. + +The future memory discovery must decide whether this is the right model, what +reopening and recovery mean at each memory layer, and how interrupted changes or +corruption are surfaced. This trace does not select immutable records, +transactions, accepted pointers, content-addressed preimages, or another repair +protocol. + +## Permanent Seam Comparison + +Scores use 1 (poor) to 5 (strong). They are preliminary source-trace analysis, +not user acceptance and not a persistence-technology score. + +| Option | Shared manual/agent operations | Session-independent truth | Non-Git recovery | Low inherited-core change | Upstream cost | Maintainer clarity | Reversible | +|---|---:|---:|---:|---:|---:|---:|---:| +| One namespaced package with integration left implicit | 4 | 5 | 5 | 5 | 5 | 4 | 5 | +| Isolated modules scattered across contracts/server/UI | 3 | 2 | 3 | 3 | 2 | 2 | 2 | +| Hybrid package plus thin integration shims and executor port | 5 | 5 | 5 | 4 | 4 | 5 | 5 | + +The hybrid scored highest in this preliminary comparison because a package can +make ownership explicit while named shims make UI/server/executor crossings +testable. It is not selected. A future memory architecture may change which +responsibilities belong together, so this comparison remains historical source- +trace evidence rather than a current package recommendation. + +## Reuse Unchanged + +- `@scientfactory/project-init` for folder inspection, initialization, + migration, and interrupted-init recovery; +- folder picker, duplicate project recovery, host project registration, and + sidebar shell; +- trusted local server and typed websocket RPC transport; +- `WorkspacePaths`/`WorkspaceFileSystem` containment for app-owned paths; +- normalized provider runtime events as execution evidence; +- provider interrupt and approval UI concepts behind the executor adapter; +- common UI primitives, toasts, loading/error surfaces, and optional file + preview; and +- existing external provider registry/settings/adapters, especially OpenCode. + +## Do Not Couple Scient Truth To + +- `ProviderKind`, model selection, or one provider's resume cursor; +- Synara project ID, thread ID, message ID, proposed-plan rows, or checkpoint + refs; +- app `state.sqlite`, browser local storage, Electron profile, or provider event + log; +- external OpenCode binary, URL, password, auth file, XDG root, session DB, or + update channel; +- Git repository presence, worktrees, branches, or hidden refs; +- unrestricted shell commands or model-generated paths; or +- T3 Code, Goose, cloud, mobile, collaboration, or remote execution. + +## Proven Gaps And Candidate Changes + +### Scient desktop candidates - not authorized + +- evaluate whether `packages/scient-project` should own domain commands, + persistence-independent interfaces, revision/recovery rules, and path-scope + policy; +- do not select persistence or memory boundaries from this trace; +- add narrow DTO/RPC contracts and server services that resolve portable + identity from the workspace root and call the package; +- add one project-level route/view and sidebar entry; and +- later evaluate the executor port, controlled deterministic adapter, and one + Scient adapter; the fake-executor product proof is deferred. + +No change to Synara's orchestration event schema, `projection_projects`, generic +provider event schema, or Git checkpoint store is required for the fixture. + +### Scient agent requirement - approved, implementation later + +Before native connection, replace the inherited hardcoded global identity/path +defaults with a build/runtime profile that gives Scient dedicated data, config, +state, cache, temp, log, auth, database, plugin, and update locations. Use a +brand-neutral durable first-party ID where records persist. Keep upstream +OpenCode defaults available to external OpenCode builds; never redirect or +migrate the user's external OpenCode state implicitly. + +After a later review, expose a bounded Scient workflow capability that consumes +the gateway receipt and returns a proposal. The first capability profile must +not provide unrestricted shell or direct canonical writes. + +## Deferred Memory-Architecture Handoff + +This trace does not authorize memory, persistence, or executor implementation. +It hands the unresolved relationships among conversation history, task/run +context, user memory, project memory, raw history, provenance, files, recovery, +and cloud sync to a dedicated future discovery project. The candidate layers, +questions, and explicit non-decisions are preserved in the +[`Idea Inbox`](../../docs/planning/idea-inbox.md#memory-context-and-continuity). + +The permanent scientific-operation package and deterministic fake-executor +product proof remain separate deferred questions. Native Scient isolation is an +approved independent requirement: native Scient and external OpenCode must not +share credentials, processes, paths, sessions, plugins, databases, or updates. + +## Go/No-Go Conditions + +The independent T3 reliability work may complete without any memory decision. +This trace also does not create a broad blocker for unrelated product work. +Memory-dependent workflow behavior should be planned only when the dedicated +memory project begins; the package, executor proof, and native-agent integration +require their own explicit authorization. + +## Review Checkpoint + +The source map is complete, but its former SQLite and hybrid-package selection +is withdrawn. The approved requirements are non-Git recovery, trusted +filesystem scope, and complete native-Scient/external-OpenCode runtime +independence. Fake-executor work is deferred. Future memory questions live in +the [`Idea Inbox`](../../docs/planning/idea-inbox.md#memory-context-and-continuity); +do not infer memory architecture or implementation authorization from this note. diff --git a/lab/notes/t3-code-targeted-review-2026-07-18.md b/lab/notes/t3-code-targeted-review-2026-07-18.md new file mode 100644 index 0000000..9c13e0a --- /dev/null +++ b/lab/notes/t3-code-targeted-review-2026-07-18.md @@ -0,0 +1,141 @@ +# T3 Code Targeted Review + +Status: Complete +Owner: Yaacov +Created: 2026-07-18 +Last updated: 2026-07-18 +Purpose: Records the bounded T3 Code inspection, accepted reliability intake, and explicit stop boundary for future T3-derived work. +Doc type: Research evidence + +## Verdict + +T3 Code remains a useful subsystem donor, not Scient's application foundation, +product roadmap, or continuously monitored upstream. The targeted review is +complete through T3 Code revision +`bf76535fe4da71d8de7b8bd5ffa0d2086b7af8d0`. + +The review found three small reliability changes worth adapting immediately, +one subscription-startup risk that Scient's current Effect runtime already +handles correctly, and one real multi-socket browser isolation defect in the +Scient desktop. Larger T3 architecture remains either part of an existing +Scient plan, deferred until a concrete trigger appears, or rejected for the +current product scope. + +## Source Snapshot + +| Item | Evidence | +|---|---| +| Official source | `https://github.com/pingdotgg/t3code.git` | +| Branch inspected | `main` | +| Review boundary | `bf76535fe4da71d8de7b8bd5ffa0d2086b7af8d0` | +| Review date | 2026-07-18 | +| License observed | MIT | +| Scient desktop base reviewed | `57e6b2cde09f64db367b894506f56db605fb91b4` | +| Review method | Exact donor commits, corresponding Scient seams, existing tests, and deterministic characterization tests were inspected locally. | + +The temporary T3 checkout used for this review is not a canonical Scient source +checkout. `lab/external/sources.lock.md` remains the durable provenance record. + +## Accepted Reliability Intake + +| T3 evidence | Scient seam | Disposition | Scient evidence | +|---|---|---|---| +| `ed81c156daf3d8ce7d3599236df7a26c11ef145f` | `apps/server/src/imageMime.ts` | Adapted. The payload parser now scans linearly instead of applying a regex to multi-megabyte base64 content. | Desktop commit `9c36410e`; focused tests cover malformed input, padding, whitespace, case handling, and a 14 MB payload from a deep stack. | +| `58302b2010913d6e236570bbe8c9abe38de7fa18` | `apps/server/src/git/Layers/GitCore.ts` | Adapted. Selected paths use Git's global `--literal-pathspecs` option. | Desktop commit `ee705427`; integration coverage includes brackets, glob characters, question marks, pathspec magic, a leading dash, spaces, and Unicode. | +| `1047dac0c5296b1c548e607f061f95227a706b49` | `apps/server/src/terminal/Layers/Manager.ts` | Adapted. AppImage markers and mount-prefixed path entries are removed at the terminal child-environment boundary. | Desktop commit `353f8d67`; tests prove marker/path cleanup, unrelated-variable preservation, non-AppImage identity behavior, and no host-environment mutation. | + +These changes preserve Scient's existing contracts and introduce no new runtime +dependency, provider model, or T3-owned abstraction. + +## Characterization Results + +### Snapshot And Live-Event Startup + +T3 commit `c14a5ca492b4da11da81e482e307222946536300` fixed events dropped while an +initial thread snapshot loaded. Scient uses `Stream.merge` for the analogous +shell and thread subscriptions. + +A deterministic barrier test now proves that Scient's current Effect runtime: + +1. subscribes to the live stream while the snapshot is blocked; +2. captures an event emitted during snapshot loading; +3. emits the snapshot first; +4. emits the during-snapshot event exactly once; and +5. continues with later live events in order. + +The characterization passed without a runtime behavior change. Desktop commit +`f12d5b56` names the shared subscription seam and adds the regression test. + +### Browser Native-Pipe Socket Isolation + +The Scient browser-use bridge attached CDP listeners per session but broadcast +every notification to every connected native-pipe socket. A two-socket protocol +test proved that a tab event for session A leaked to session B. The same test +also showed that disconnecting a socket left its CDP listener active. + +Desktop commits `fe4668fa` and `8a8398e8` bind each attached session to its +owning socket, send notifications only through that socket, reject cross-client +attach, detach, and CDP requests, dispose the session listener on disconnect, +and preserve the other socket. The test exercises real framed native-pipe +requests and two distinct tabs rather than only testing an internal helper. The +complete bounded intake passed hosted CI at exact head `8a8398e8` and merged as +`bd2a6eed` through +[Scient desktop PR #14](https://github.com/ScientFactory/scient-desktop/pull/14). + +## Use During Existing Scient Work + +T3's provider-instance separation is useful evidence for Phase 2 of +`docs/planning/scient-and-external-agents-implementation-plan.md`. It does not +justify a separate T3-alignment project. + +When the first scientific vertical slice reaches its execution contract: + +- keep `ProviderKind` as an external-provider implementation detail; +- add stable agent-connection identity above it; +- represent native Scient and external-agent connections as distinct execution + targets; +- later evaluate one narrow executor port; Yaacov deferred the deterministic + fake-executor product proof; +- preserve existing external adapters and settings; and +- keep canonical scientific state independent of every provider session store. + +The independent T3 intake is complete and has no dependency on the future +memory-architecture project. Raw questions about conversation, user, project, +and task/run memory; portability; recovery; Git and cloud folders; and future +Scient cloud sync are preserved separately in the +[`Idea Inbox`](../../docs/planning/idea-inbox.md#memory-context-and-continuity). + +## Triggered Shelf + +The following ideas are not scheduled. Reinspect T3 only when the named trigger +exists. + +| Candidate | Trigger required before work | First bounded proof | +|---|---|---| +| Connection supervisor | More than one real connection or recurring lifecycle leaks | Tested connection state machine | +| Desktop-main decomposition | Repeated feature collisions in the same desktop lifecycle seam | Extract only the seam under active change | +| Diagnostics bundle | A user-visible failure cannot be diagnosed from existing logs | Redaction-first diagnostic export | +| Reusable Codex protocol package | A second genuine protocol consumer | Extract only proven shared protocol types | +| VCS/forge abstraction | The scientific slice requires a demonstrated collaboration operation | Model that operation only | +| Execution environments or SSH | An accepted remote-execution scenario | One target-specific adapter | +| Capability authorization | An executor gains a new privileged action | Explicit capability and denial tests | + +## Rejected For Current Scope + +- broad T3 architecture alignment; +- replacing the Synara-derived desktop foundation; +- importing T3's coding-product assumptions; +- mobile companion work; +- cloud or relay synchronization; +- Tailscale integration; +- WSL expansion; and +- speculative SSH or remote-environment infrastructure. + +## Operating Rule After This Review + +Do not monitor T3 Code as though it were an owned upstream. Future T3 inspection +must begin with a concrete Scient problem, name the exact source revision and +files inspected, produce a bounded adaptation or explicit rejection, and stop +when that problem is resolved. + +The primary product track returns to the first scientific vertical slice.