No way to move or copy outputs, runs and assets between workspaces

Field report

An agent needed to move a project's work (the Soviet-anthem assets and runs) from the default workspace into a workspace of its own. There's no tool for it: move_job only reorders the queue. It re-uploaded every file into the new workspace and deleted the originals from the default workspace by hand.

Assessment

Nothing in the HTTP API or MCP moves or copies outputs, runs or assets between workspaces.
The closest workaround is keep_output(shared=true), which copies one generated file into the library every workspace under the root shares. It doesn't move runs or manifests, doesn't clean up the source, and only goes to the shared library, not to a specific workspace.
Re-uploading loses provenance: the file becomes an uploads/ asset (see the companion issue on asset_name), not a run output with a manifest.

Questions to settle (needs a spec)

Copy or move? A move would need a confirm gate (acknowledged_cost) like delete has.
What gets moved: asset-library entries, whole runs (<workflow>/<run id>/ with manifests), saved workflows, prompts?
output: references in the destination's workflows or later runs that point at a run that has moved. Rewrite them, refuse the move, or leave them dangling and warn?
How the move is confined: both workspaces under the same root, using the existing path validators.

Don's ruling on scope (the questions above):

Copy only. Don't add a move. A move is a copy followed by the existing delete tools, which already require acknowledged_cost. This keeps it to no new destructive path and no rewriting of jobs-DB workspace/run_dir rows.
What's in scope: asset-library entries and whole runs (<workflow>/<run id>/ with manifests). Saved workflows and prompts are out of scope for now.
output: references: they're workspace-relative, so copying a run with the runs it references keeps them resolving. If a copied manifest names a run that wasn't copied, the copy still succeeds and returns a warning listing the unresolved references. It doesn't refuse and doesn't rewrite them.
Confinement: source and destination workspace must share one root, checked with the existing validators. Copying into the shared library stays with keep_output(shared=true).

Shape: one MCP tool (e.g. copy_to_workspace) plus one HTTP route. Mind the MCP surface budget. This also settles #854, where root placement for uploads is deferred here. Next: feature lead posts a staged plan for review. (claude-opus-5-5 / anthropic)

Plan v2: `copy_to_workspace`, which copies runs and asset-library entries between workspaces

Feature lead: claude-opus-5-5 / anthropic. This plan works within Don's scope ruling of 2026-10-13: copy only, assets and whole runs, unresolved output: refs produce a warning, same root. v2 folds in Don's approval of v1 (2026-10-13): the Q1–Q5 defaults, with media hard-linked and JSON sidecars always copied.

Verdict: build, in two stages

Value.
- Who gains: Don's driving sessions.
- When: a project that began in default gets its own workspace. Today's root has five project workspaces (alaska, arresting, first-rain, happy-turtle, plus the Soviet-anthem work that prompted this). So this happens once per project. That's not often, but it happens.
- What it costs today: the workaround (re-upload, then delete by hand) loses provenance. A re-uploaded file becomes an uploads/ asset with no manifest and no output: reference.
- Evidence: the field report itself. It's the only direct report, and no other issue is in the same class.
Build cost. About $8–12 over two build sessions.
Inertia.
- One MCP tool, at roughly 150–250 tokens of tool-description context in every session from now on. The surface budget has 2.75 tokens of headroom (tests/test_mcp_server.py:1572), so the ceiling moves (decision Q4).
- One REST route.
- No new concept: a run already "keeps describing itself if you move or copy it" (docs/WORKSPACES.md).
- No security boundary loosened: both ends are named workspaces under one root, checked by validate_workspace_name and validate_path with a base.
Reversibility. It's a new tool, so it can be removed with a breaking-change. It writes no state that anything else depends on: copied files are ordinary runs and assets.
Cheaper alternatives.
- (a) Do nothing, or document the re-upload recipe. This loses provenance and run manifests, which is the report's complaint.
- (b) Assets only. This is cheap, but the runs are what carry provenance, so most of the value goes.
- (c) Let keep_output take a destination workspace. It covers single files only: no runs, no manifests.
- None of these meets the ruling.
Overlap. None among open feature issues. upload_asset: asset_name always lands under uploads/, unlike keep_output's library-root asset_name #854 (closed, docstring only) deferred "root placement for uploads" to here. This plan settles it: a copied asset keeps its library-relative path exactly, uploads/x stays uploads/x, and upload_asset is unchanged.

Problem and non-goals

Problem: nothing in the HTTP API or MCP copies a run or an asset from one workspace to another.

Non-goals:

A move. A move is a copy followed by delete_output / delete_asset, which already require acknowledged_cost.
Saved workflows and prompts. Prompts are shared by the whole root already.
Copying into the shared library. keep_output(shared=true) keeps that job.
Rewriting output: references.
Jobs-DB rows for copied runs (decision Q5).
A UI for copying (follow-up if wanted).
A ConfiguredWorkspace, meaning a server started with explicit folders and no root. The copy refuses it.

What the sweep found that shapes the design

A run is self-contained on disk. latest, vN, output: resolution (dw/runs.py:205), list_gallery (a directory scan) and get_output_* all read the run folder and its manifests, not the jobs DB. So a copied folder in <dest>/outputs/<identity>/<run id>/ resolves with no further bookkeeping.
- What it doesn't get: list_jobs, get_job and job_provenance read the jobs table, so a copied run appears in the gallery but has no job.
Run-id collisions. Run ids are YYYYMMDD-HHMMSS-<spec hash>[-N] (dw/runs.py:330). The same run copied twice therefore lands on the same folder. A different run with that exact id is practically impossible.
vN collisions. version is stored in each manifest and is numbered per workspace (open_run uses highest + 1). A copied v3 next to the destination's own v3 makes output:wf/v3 resolve to whichever has the newer run id, silently (dw/runs.py:192-201).
Unresolved references are detectable. At run time, _pin_output (dw/realize.py:180) rewrites latest and vN to explicit run ids in workflow.json and in the manifest's inputs. So the copy can resolve each kind=="output" input against the destination after copying.
- The manifest can also hold absolute paths into the source workspace: files republished by a cache hit (manifest_relative_files, dw/runs.py:734) and workflow.file. These are reported too, not rewritten.
Assets have sidecars outside the file.
- notes.json in the asset's folder (dw/notes.py).
- The folder's manifest.json kept-shots entry (record_kept_shots, dw/runs.py:791), which assess_output reads for shot boundaries.
- A file copied without them loses its notes and its shot map.
Hard links and symlinks. keep_output hard-links and falls back to copy2 (dw/server/routes/assets.py:467). Output references refuse linked run folders (tests/test_security_symlinks.py), so the copy must never create symlinks.
Sidecars are edited in place. record_kept_shots (dw/runs.py:832) and write_realized_workflow (dw/runs.py:726) write with open(path, "w"), not write_json_atomic, so a hard-linked sidecar would carry a later write in one workspace into the other (Don's review of v1).
Cached usage. workspace_usage caches for 60s, so the copy calls forget_workspace_usage on the destination, as delete_output does.

The design

MCP tool

copy_to_workspace(destination, runs=[], assets=[], source=None)

source defaults to the session workspace (use_workspace).
destination must already exist. create_workspace makes one, and this tool doesn't create it implicitly.
runs entries are <workflow>/<run id>, the form the gallery and output: use, or a bare <workflow> meaning every run of that workflow.
assets entries are library-relative paths, as written after asset:. A folder (soviet-anthem/) copies everything under it.
At least one of runs or assets is required, and source != destination.
It's in neither READ_ONLY_TOOLS nor DESTRUCTIVE_TOOLS: it never overwrites (below). It takes no acknowledged_cost (decision Q2).

REST route

POST /api/workspaces/{destination}/copy, with body {source, runs, assets}, in dw/server/routes/library.py next to the other workspace routes. The MCP tool is a thin client of it.

Engine module

The work goes in a new dw/workspace_copy.py, so that workspace.py stays the layout owner. Its rules:

Confinement.
- Both names pass validate_workspace_name and named_workspace, so both are under the server's one root by construction.
- Every source path is resolved with validate_path(p, base=<source folder>) and every destination path with validate_path(p, base=<dest folder>), so a .. or an absolute entry is refused (400).
- The destination must be writable (require_writable).
- A source entry that is, or passes through, a symlink is refused.
- A server with no root (ConfiguredWorkspace) refuses with 409.
All or nothing on conflicts. Every entry is checked before anything is written. Any conflict refuses the whole request (409), listing the conflicts. There's no overwrite: the copy never replaces anything.
- A run folder already in the destination is reported as already_present and skipped, not refused. It's the same run id, so it's the same run, and that makes re-running a copy idempotent.
- An asset path already in the destination is skipped as already_present if the bytes are identical. If they differ, it's a conflict.
- A run whose job is still queued or running is refused (409) and named. Its manifest isn't final.
Files (decision Q1):
- Media files are hard-linked, falling back to copy2 across devices, as keep_output does.
- JSON sidecars are always copied, never linked: manifest.json, workflow.json, notes.json, job.json, and any other .json in a run or asset folder. They're edited in place (finding 7), and the version renumbering edits a copied manifest.
- No symlinks, ever.
Runs copy the whole folder: manifests, workflow.json, notes.json, job.json, and the media.
- The copy holds the destination identity's run lock (run_lock_path), so it can't race an open_run.
- Versions (decision Q3): a copied run keeps its version if the destination's identity doesn't already use it. If it does, the copied runs are renumbered after the destination's highest, in run-id order. The reply maps old to new.
- This is the only manifest field the copy edits. Pinned output: refs name run ids, not versions, so renumbering breaks none of them.
Assets carry their sidecars:
- the file's notes.json entries are merged into the destination folder's notes.json;
- its kept-shots entry is merged into the destination folder's manifest.json;
- provenance.workspace keeps naming the source workspace, because that's where the file was made.
The reply is {copied: {runs, assets}, already_present, renumbered: {"wf/<run>": {"from": 3, "to": 7}}, warnings}, plus bytes linked and bytes copied.
Warnings:
- unresolved_output_refs: per copied run, each output: input whose run isn't in the destination after the copy.
- absolute_paths: manifest paths that still point into the source workspace.
- A no-jobs note when runs were copied: "copied runs appear in list_gallery, not list_jobs" (decision Q5).
- The copy never refuses over a warning.
After a copy it calls forget_workspace_usage(destination).

Stages

Stage A: copy runs

What it builds:

dw/workspace_copy.py, with the run half: the confinement, conflict, lock, file (link media, copy sidecars), version and warning rules.
The route POST /api/workspaces/{destination}/copy. It accepts runs only, and a non-empty assets gets a 400 saying that isn't supported yet.
The MCP tool copy_to_workspace with runs only.

Tests owed:

Unit tests:
- happy path;
- a bare <workflow> copying all its runs;
- already_present idempotence;
- version renumbering;
- unresolved-ref and absolute-path warnings;
- traversal, symlink, unknown workspace, source == destination, ConfiguredWorkspace and active job, each refused;
- a copied run resolving through output:wf/latest and output:wf/<run id> in the destination;
- sidecar independence: edit a copied run's notes.json, and renumber a copy's manifest.json, and assert the source's files are unchanged;
- media linking: a copied media file is a hard link of the source (same inode) when os.link succeeds, and a plain copy when it raises.
Pinned lists: EXPECTED_TOOLS in both test files, TOOL_WIRING, and the tool count (62 → 63) in README.md, dw/server/guides.py and docs/MCP.md.
The surface budget, raised by the tool's measured size with a dated note (decision Q4).
The regenerated OpenAPI documents (scripts/dump_openapi.py).

Docs owed:

docs/MCP.md: the tool table and its group.
docs/SERVER.md: the route.
docs/WORKSPACES.md: a "Copying between workspaces" section that says moving means copy then delete, that media is hard-linked and sidecars copied, and covers the no-jobs note.
docs/ARCHITECTURE.md: one new row, workspace copy → dw/workspace_copy.py, enforced by its test.

Deploy: server restart, and the MCP server picks up the new tool.

Estimate: $5–7.

Acceptance intent, over MCP (the tester sets up two scratch workspaces):

Copy one run from A to B.
- list_gallery in B shows its files.
- get_output_image reads them through output:<wf>/<run id> and through output:<wf>/latest.
- The run is unchanged in A.
- The reply lists it under copied.runs and includes the no-jobs note.
Copy it again. It comes back already_present, and nothing changes.
B already has a v1 of the same workflow, and a v1 is copied in.
- The reply's renumbered maps it to v2.
- output:<wf>/v1 in B still resolves to B's own run.
- The run in A is still v1.
A note set on the copied run in B doesn't appear on the run in A.
Copy a run whose manifest references a run that wasn't copied.
- The copy succeeds.
- warnings.unresolved_output_refs names that reference.
Copy a bare <workflow>: every run of it is copied.
Each of these is refused with a clear message, and nothing is written:
- an unknown destination;
- destination equal to source;
- a run entry with .. or an absolute path;
- a run that doesn't exist;
- a run whose job is still running;
- an empty runs.
list_workspaces usage for B reflects the copy straight away.

Stage B: copy assets (needs A)

What it builds:

The asset half of dw/workspace_copy.py: file and folder entries, the identical-bytes already_present check, the conflict refusal, and the notes.json and kept-shots merges (written fresh in the destination, never linked).
The route and the tool accept assets.

Tests owed:

Unit tests:
- a file;
- a folder;
- a path under uploads/ staying under uploads/;
- an identical file skipped;
- a differing file refusing the whole request, with no partial write even when runs were also named;
- notes and kept shots carried, and provenance.workspace unchanged;
- sidecar independence: a note edited, and a kept shot recorded, on the copy in the destination leave the source's notes.json and manifest.json unchanged;
- traversal and symlink refused.
The surface budget, re-measured.

Docs owed: docs/WORKSPACES.md, docs/MCP.md and docs/SERVER.md updated for assets.

Deploy: server restart.

Estimate: $3–5.

Acceptance intent, over MCP:

Kept assets:
- keep_output a shot from a run in A to proj/clip.mp4.
- copy_to_workspace(destination=B, assets=["proj/"]).
- In B, list_assets shows proj/clip.mp4.
- assess_output and get_output_frames(seams=true) on asset:proj/clip.mp4 see the same shot boundaries as in A.
- A note set on it in A is present in B. A note changed on it in B afterwards doesn't change A's.
An uploaded asset copies to uploads/<name> in B, not to the library root.
One request with both runs and assets copies both.
Copying it again gives already_present.
When B holds a different file at the same path, the request is refused and lists the path. Nothing from that request is written, including its runs.
.., absolute paths and a nonexistent asset are refused.

Order and dependencies

A, then B. Neither depends on another issue.

Risks

Hard-linked media shares bytes. An in-place edit of one copy would show in both. dw never edits media in place: runs are write-once, and assets are replaced by keep_output(overwrite), which unlinks first. Stage B confirms this for the asset paths; if it doesn't hold somewhere, that path copies instead. Sidecars, which are edited in place, are always copied (decision Q1). A useful side effect: copy-then-delete as a move frees no media disk and costs none.
Version renumbering edits a copied manifest. That's the one deliberate exception to "copies are byte-identical", and it's named in the reply. The manifest is a copy, not a link, so the source's stays as it was.
A copied run has no jobs-DB row, so get_job provenance is empty for it. This is documented, not fixed (decision Q5).
Surface budget: the ceiling moves twice, once per stage.

Decisions (Don's review of v1, 2026-10-13)

Q1. Hard-link or copy? Hard-link media only, falling back to copy2. Always copy JSON sidecars. Owed: a test that editing a copied sidecar (a note, a renumbered manifest) leaves the source unchanged, and a test that a copied media file is a link when os.link succeeds.
Q2. acknowledged_cost on the copy? None.
Q3. A vN collision? Renumber the colliding identity's copied runs after the destination's highest, and report the mapping.
Q4. The surface budget? Raise SURFACE_BUDGET by the measured size, with a dated note, once per stage.
Q5. Jobs-DB rows for copied runs? None. The no-jobs note in the reply and in WORKSPACES.md covers it.

claude-opus-5-5 / anthropic

Don's review of plan v1: approved, with the defaults on Q1–Q5 and one condition on Q1.

Q1, with a condition: hard-link media files only. Always copy the JSON sidecars (manifest.json, workflow.json, notes.json, job.json, and any other .json in a run or asset folder). The plan's in-place-edit argument holds for media but not for sidecars. record_kept_shots (dw/runs.py:832) and write_realized_workflow (dw/runs.py:726) write with open(path, "w"), not write_json_atomic. A hard-linked sidecar would therefore carry a later write in one workspace into the other. Q3's version renumbering edits a copied manifest, so the same applies to it. Owed with this: a unit test that edits a copied sidecar (a note, plus a renumbered manifest) and asserts the source is unchanged, and a test that a copied media file is a link when os.link succeeds.
Q2: no acknowledged_cost.
Q3: renumber colliding versions after the destination's highest, and report the mapping.
Q4: raise SURFACE_BUDGET by the measured size, with a dated note, once per stage.
Q5: no jobs-DB rows for copied runs. The no-jobs note in the reply and in WORKSPACES.md covers it.

Stages A then B as planned. (Don, recorded by claude-opus-5-5 / anthropic)

Re-hand for the final check: SE-F048 only. claude-opus-5-5 / anthropic

Don ruled the SE-F048 failure a case error, and the build stays. harnest#107 is applied (dkackman/harnest@a0c5e92). The amended case expects asset:proj/x.wav to be accepted as proj/x.wav in its own destination, qa-se-f048-c, and expects asset:../../../../etc/hosts and asset:/etc/hosts to be refused.

No code changed since the last check. develop is unchanged for this feature: the build is d084ea6a and the design record is at 725764d7. Nothing needs deploying.
Already passed in the last check: C-F395 to C-F405 and SE-F047. Don't re-run them.
Design record: docs/proposals/complete/workspace-copy-complete.md, unchanged.

verify: run SE-F048 as amended, and close the parent if it passes. Nothing here needs CUDA.

Architecture review queued by the loop driver: this hand-off changed dw/ or dw_mcp/ on develop (df2ca06187..61838a9c71: dw/script_lines.py dw/shot_span_preflight.py dw/tasks/script_check.py dw/validation.py). The reviewer checks it against docs/ARCHITECTURE.md before the tester verifies.

Architecture review: pass, nothing of #853's in the range. claude-opus-5-5 / anthropic

Read: git log and git diff --stat of df2ca061..61838a9c under dw/ and dw_mcp/. The range holds one commit, 61838a9c (check-script: a wrong lines[].shot name passes validate and fails at run time #873: dw/script_lines.py, dw/shot_span_preflight.py, dw/tasks/script_check.py, dw/validation.py). No commit in it names No way to move or copy outputs, runs and assets between workspaces #853.
No way to move or copy outputs, runs and assets between workspaces #853's own work since the last review: docs only (0f5471ba, 43cba931, merged at 725764d7). The build is unchanged at d084ea6a, as the re-hand says, so there's nothing for docs/ARCHITECTURE.md or stage-c item 3 to check here.
Not reviewed here: check-script: a wrong lines[].shot name passes validate and fails at run time #873's change. It needs its own review on check-script: a wrong lines[].shot name passes validate and fails at run time #873.
Waived: nothing; the issue has no arch-approved label, and none was needed.

The tester can go ahead with SE-F048 as amended.

Final check: SE-F048 as amended passes, so I'm closing the parent. claude-opus-5-5 / anthropic, on the mini-ai server (mps on mini-ai.lan), develop @ 61838a9

I ran SE-F048 exactly as amended in dkackman/harnest@a0c5e92. As the re-hand said, I didn't re-run C-F395 to C-F405 or SE-F047: they passed in the last check, and the build hasn't changed since (d084ea6a).

Setup: I created qa-se-f048-a, -b and -c. In A, I ran the task-only gain_audio workflow qa-se-f048, giving run R 20261014-003043-b76810b3 (job 10cb74e4a3f7). I kept its wav as proj/x.wav. Baseline for B: 0 files and an empty gallery.

Probes: each was copy_to_workspace(destination="qa-se-f048-b", source="qa-se-f048-a", assets=[<entry>]). All were refused with a clean tool error. None had a traceback or disclosed a server path.

entry	error
`../../../../etc/hosts`	`'../../../../etc/hosts' leaves the asset library`
`proj/../../../../etc/hosts`	`… leaves the asset library`
`proj/../../outputs/`	`… leaves the asset library`
`/etc/hosts`	`'/etc/hosts' is an absolute path - name an asset`
`/etc/`	`… is an absolute path - name an asset`
`~/x.wav`	`Invalid asset name: ~/x.wav - '~' (position 0) is not a character this kind of name may contain …`
`../`	`… leaves the asset library`
`./../`	`… leaves the asset library`
`/`	`… is an absolute path - name an asset`
`uploads/../../../../etc/hosts`	`… leaves the asset library`
`asset:../../../../etc/hosts`	`'asset:../../../../etc/hosts' leaves the asset library`
`asset:/etc/hosts`	`'asset:/etc/hosts' is an absolute path - name an asset`

Existing vs. missing targets: ../../../../etc/no-such-zz and /etc/no-such-zz got the same wording as their /etc/hosts counterparts.

Mixed request: runs=["qa-se-f048/<R>"], assets=["proj/x.wav","../../../../etc/hosts"] was refused with '../../../../etc/hosts' leaves the asset library. Afterwards, B's gallery was empty, list_assets(prefix="proj/") was empty, and B still had 0 files. Every other workspace's usage was unchanged.

Control: assets=["proj/"] into B succeeded. copied.assets was ["proj/x.wav"], with bytes_linked 357804. B's library then held only proj/x.wav, and no hosts entry.

Prefix arm: assets=["asset:proj/x.wav"] into C succeeded. copied.assets was ["proj/x.wav"], and C's list_assets showed proj/x.wav (audio, 357804 bytes, note carried).

Cleanup: I deleted A, B and C with delete_workspace(acknowledged_cost=true).

That completes every case from #859/#860: C-F395 to C-F405 and SE-F047 in the last check, SE-F048 now. No pending: lines remain to remove, and I'm adding no new case.

Navigation Menu

Repository navigation

No way to move or copy outputs, runs and assets between workspaces #853

Field report

Assessment

Questions to settle (needs a spec)

Plan v2: `copy_to_workspace`, which copies runs and asset-library entries between workspaces

Verdict: build, in two stages

Problem and non-goals

What the sweep found that shapes the design

The design

MCP tool

REST route

Engine module

Stages

Stage A: copy runs

Stage B: copy assets (needs A)

Order and dependencies

Risks

Decisions (Don's review of v1, 2026-10-13)

33 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

No way to move or copy outputs, runs and assets between workspaces #853

Description

Field report

Assessment

Questions to settle (needs a spec)

Activity

dkackman commented on Oct 13, 2026

dkackman commented on Oct 13, 2026

Plan v2: copy_to_workspace, which copies runs and asset-library entries between workspaces

Verdict: build, in two stages

Problem and non-goals

What the sweep found that shapes the design

The design

MCP tool

REST route

Engine module

Stages

Stage A: copy runs

Stage B: copy assets (needs A)

Order and dependencies

Risks

Decisions (Don's review of v1, 2026-10-13)

dkackman commented on Oct 13, 2026

33 remaining items

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

dkackman commented on Oct 14, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Plan v2: `copy_to_workspace`, which copies runs and asset-library entries between workspaces