Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,32 @@ are in [docs/plan.md](docs/plan.md) §12. Upgrading from v0.3: see the

## Unreleased

- **Audit batches 33 and 34a: what is published is what was answered, of the image checked.**
*Yes, these are right* saves exactly the tags a block shows (a hidden proposal is never
saved), and a unit tagging one label twice is refused before it is saved
(`E_DUPLICATE_CHANNEL_IN_UNIT`). An unanswered image type is an open question
(`unit_image_type_unanswered`; "channels" is still published as an absent `image_type`).
`publish_draft` now requires `channel_counts`, each unit's channel count read through the
confirmed axis roles: tags saved for another count (`unit_channels_stale`) or a unit without
a count (`unit_channel_count_unknown`) are refused. The notebook's folder check binds every
header to its file (`bind_headers=True`); the tag pictures, the Step 1 pictures and the mask
evidence are drawn only of that revision (masks paired again without a new check are bound
too, and evidence already read is not shown once its image or mask changed), and the save is
refused when an image changed since the check. The axis step's preview is not bound yet.
*Check and save* shows tags saved for another channel count as stale, not as the folder's
channels. An answer the draft file cannot take (lock, write error) is
refused and not shown as answered. `make_described_run` refuses a member with no technical
repeat or fewer images than technical repeats before writing anything. A new record the
records reader would refuse is never written (`R_RECORDS_FILE_INVALID`), and a run refused
for its records file reports that code with its `E_RECORDS_*` reasons instead of
`description_unreadable`. `all_colour` counts a unit only when its tags are exactly R, G, B.
**API changes:** `publish_draft(..., channel_counts=...)` is a new required keyword;
`missing_confirmations` and `description_from_draft` take an optional `channel_counts`;
`header_preview(..., bound=...)` is a new required keyword; `mask_evidence` takes
`options: StripOptions, bound` in place of `cache, memory_limit`; `Thumbnails` takes the
check's `header_fingerprints`. A draft saved before this change with no image type reopens
with the image type asked, never as "channels".

- **Notebook: folder pictures with a 1:1 crop, "Tag like the saved group", colour mask evidence
(PR 17c).** *Images found* adds a picture of each folder's first image to the labels table,
from the file header only and display only (`header_preview`, `notebook_folder_previews`):
Expand Down Expand Up @@ -52,7 +78,9 @@ are in [docs/plan.md](docs/plan.md) §12. Upgrading from v0.3: see the
(`inconsistent_mapping`, `missing_axes`, `ambiguous_missing_axes`) no longer advise exporting the
images the same way: they stay errors and say they follow from the layouts. The notebook's axis
step says that no roles are suggested because the units declare different layouts (fluorescence
and RGB). The refusal codes are unchanged: nothing refused before is accepted.
and RGB). A previously blocking input stays blocked; diagnostic codes may change (a described
run reports the per-type checks and the Spec's codes in place of the global `mixed_dtypes` and
`inconsistent_dimensions`).
- **Notebook: one step at a time (UX proposal 2026-11-09, PR 17a).** `PreprocessingNotebook.steps()`
shows the shipped notebook's 13 steps one at a time, with a strip of chips (done green with a tick,
current blue, to do grey, changed by a reopen orange; a chip opens its step) and Back/Next (Next is
Expand Down
9 changes: 6 additions & 3 deletions docs/contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,10 @@ planes of an experiment use the same bin count. In a described run (a published
`morphobricks.toml`) the pixel type is compared among the units that declare the same
`image_type`, and the notebook's folder check compares images inside each unit; units of
different image types (RGB and fluorescence) are refused as `R_HETEROGENEOUS_LAYOUT_NOT_ENABLED`
(Stages 1-2), not as `mixed_dtypes`. An undescribed run compares every image of the experiment.
(Stages 1-2), not as `mixed_dtypes`. An undescribed run compares every image of the experiment. The
invariant is that a previously blocking input stays blocked; the diagnostic codes may change
(the per-type checks and the Spec's `R_*` codes replace the global `mixed_dtypes` and
`inconsistent_dimensions` of a described run).

## 4. Histogram semantics

Expand Down Expand Up @@ -687,5 +690,5 @@ Rules.
- **Saving** (`save_draft`) is atomic (temporary file, one replace) and under the output folder's lock, after each answer. The answers are applied to the draft and the exact bytes are read back with the strict reader first, so an answer that would be refused on resume (a `spec_commit` other than the pinned one, `spec_commit_mismatch`; an ID the reader would refuse, a folder the draft does not have, a date that is not a real day) is refused when saved and nothing is written. An incomplete draft can be saved. The file is replaced, not merged. A symbolic link at the draft's path is refused and never replaced.
- **Reading** is strict and never repairs: bytes that are not UTF-8 JSON, a byte-order mark, a duplicate key, `NaN`, another `format`, an unknown `version`, an unknown key at any level, a missing key, a value of the wrong type or a date that is not a real `YYYY-MM-DD` day are `DraftFileError` (`code`: `malformed`, `unknown_version`, `unknown_key`, `R_DRAFT_BASE_MISSING`, `R_FILE_SYMLINK`). Reading accepts any 40-digit `spec_commit`; whether it is the current one is decided at resume and save, below.
- **Resume** (`load_draft`): the file's `spec_commit` must equal the pinned `SPEC_COMMIT`, else `DraftFileError` with code `spec_commit_mismatch` (owner decision, 2026-11-08): the confirmations in the file were given under other rules, so none is carried forward. The message names both commits, says the published `morphobricks.toml` is untouched and says to start a fresh draft from the folders. Library code never rewrites the stored commit and never deletes or replaces the stale draft; the caller derives a fresh draft and saves it, which replaces the file. `publish_draft` resumes first, so it refuses the same way and writes nothing. Then the file's `base_sha256` must equal the SHA-256 of the file published now (or `absent` if there is none), else `R_PUBLISHED_FILE_CHANGED` (`PublicationRefused`); the draft is never merged with the new file, it is derived again from it. An answer keyed by a folder or repeat the draft no longer has is `DraftError` (`unknown_key`), never moved to another folder.
- **Draft to description** (`description_from_draft(draft, answers)`, pure): refuses with `DraftIncomplete`, listing **every** missing confirmation once (`id_pending`, `repeat_not_confirmed`, `confirmed_on_missing`, `control_unresolved`, `legacy_folder_missing`, `unit_channels_missing` for a unit with no channel list, `unit_channel_untagged` for a position still `null`, `channel_undeclared` for a channel a unit names that is not declared), otherwise returns the `Description`. Nothing is confirmed for the user. `used_ids` is the draft's retired IDs plus every ID confirmed now. The description is then validated by the strict reader at publication; the function does not repeat the reader's rules.
- **Publication** (`publish_draft`): under one hold of the lock the draft is resumed, the description built, published with all the checks of `publish_description` (symbolic link, base, transition, strict reader on the exact bytes), and then the draft file is deleted. Any refusal writes nothing: the draft and the published file stay byte for byte as they were. If the draft cannot be deleted after a successful publication, the error is raised and the publication stands; the leftover draft records a base that no longer matches and is refused on resume.
- **Draft to description** (`description_from_draft(draft, answers, *, channel_counts=None)`, pure): refuses with `DraftIncomplete`, listing **every** missing confirmation once (`id_pending`, `repeat_not_confirmed`, `confirmed_on_missing`, `control_unresolved`, `legacy_folder_missing`, `unit_channels_missing` for a unit with no channel list, `unit_image_type_unanswered` for a unit whose image type is not answered `channels` or `rgb` (an old draft without one reopens with the question asked, never as `channels`), `unit_channel_untagged` for a position still `null`, `channel_undeclared` for a channel a unit names that is not declared; with `channel_counts`, the channel count of each unit's image now read through the confirmed axis roles, also `unit_channels_stale` for tags saved for another count and `unit_channel_count_unknown` for a unit without a count), otherwise returns the `Description`. A unit tagging one label at two positions is refused by `apply_answers` before anything is saved (`E_DUPLICATE_CHANNEL_IN_UNIT`). Nothing is confirmed for the user. `used_ids` is the draft's retired IDs plus every ID confirmed now. The description is then validated by the strict reader at publication; the function does not repeat the reader's rules.
- **Publication** (`publish_draft(..., channel_counts=...)`, the counts required): under one hold of the lock the draft is resumed, the description built with the counts, published with all the checks of `publish_description` (symbolic link, base, transition, strict reader on the exact bytes), and then the draft file is deleted. Any refusal writes nothing: the draft and the published file stay byte for byte as they were. If the draft cannot be deleted after a successful publication, the error is raised and the publication stands; the leftover draft records a base that no longer matches and is refused on resume.
9 changes: 7 additions & 2 deletions src/morphobricks_preprocessing/channel_answers.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,17 @@ def all_colour(layouts: Sequence[UnitLayout], units: Mapping[UnitKey, UnitAnswer


def _declared_colour(layout: UnitLayout, answer: UnitAnswer | None) -> bool:
"""Return True for a readable unit with a saved ``rgb`` answer that still fits its image."""
"""Return True for a readable unit with a saved ``rgb`` answer that still fits its image.

The tags must be exactly ``R``, ``G``, ``B`` in this order (experiment-v1 section 4.4b): a
stale or malformed answer (a resumed draft with other tokens) never counts.
"""
return (
layout.usable
and answer is not None
and answer.image_type == "rgb"
and layout.channels == len(RGB_IDS) == len(answer.channels) # a stale answer never counts
and layout.channels == len(RGB_IDS)
and tuple(answer.channels) == RGB_IDS
)


Expand Down
5 changes: 4 additions & 1 deletion src/morphobricks_preprocessing/channel_labels.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,9 @@ class UnitLayout:
``image`` is the first image of the unit by sorted path, whose header gives ``channels`` (the
number of C indices, 3 for a colour image) and ``names`` (the raw header names by C index,
None where it has none). ``problem`` is why the unit cannot be tagged (no readable first
image, roles that do not fit it); then ``channels`` is 0.
image, roles that do not fit it); then ``channels`` is 0. ``uniform`` is False when some
other image of the unit has another header signature (channel count, raw names or colour):
such a unit is never put in a block with others.
"""

key: UnitKey
Expand All @@ -214,6 +216,7 @@ class UnitLayout:
rgb: bool = False
problem: str = ""
images: int = 0
uniform: bool = True

@property
def usable(self) -> bool:
Expand Down
20 changes: 14 additions & 6 deletions src/morphobricks_preprocessing/channel_units.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,9 @@ def unit_layouts(
"""Return one layout per unit key, from the unit's first image by sorted path.

A unit whose first image cannot be read, or does not fit the confirmed roles, is returned
with a ``problem`` and no channels: the next image is **not** used instead.
with a ``problem`` and no channels: the next image is **not** used instead. A unit whose
other images do not all have the first one's header signature is ``uniform`` False, so the
tag grid never puts it in a block with other units on its first image alone.
"""
images = unit_images(discovery, excluded)
headers = {f.relative_path: f for f in metadata.files}
Expand All @@ -90,12 +92,18 @@ def unit_layouts(
)
)
continue
out.append(
UnitLayout(
key, first, count, tuple(meta.raw_channel_names), meta.rgb_samples, "", len(found)
)
)
signature = (count, tuple(meta.raw_channel_names), meta.rgb_samples)
uniform = all(_signature(headers.get(i), review) == signature for i in found[1:])
out.append(UnitLayout(key, first, *signature, "", len(found), uniform))
return tuple(out)


def _signature(meta: TiffMetadata | None, review: AxisReview) -> tuple | None:
"""Return an image's header signature (count, raw names, colour); None when unreadable."""
if meta is None or not meta.readable:
return None
count = channel_count(meta, review)
return None if count is None else (count, tuple(meta.raw_channel_names), meta.rgb_samples)


__all__ = ["channel_count", "header_names", "slot_count", "unit_images", "unit_layouts"]
Loading
Loading