Skip to content

Record completeness and segmentation snapshot provenance in mask_relationships.json #166

Open

Description

@guijacquemet

Problem

A mask scan can legitimately run while segmentation is incomplete because missing masks are warnings, not errors. However, mask_relationships.json currently looks like a final experiment-level description even when only part of the experiment has masks.

There is also a wording/provenance mismatch: the notebook says Describe the masks describes the recorded run, but the scan performs a fresh preflight and reads the masks currently present. A valid mask may therefore have changed or been added since the preprocessing manifest.

For live/incremental segmentation we need to know exactly what snapshot the report describes.

Suggested simple implementation

Make the report self-describing.

Record at least:

  • total analysed experiment FOVs;
  • FOVs with any masks;
  • per mask type: masks present / masks expected / missing;
  • for each pairwise relationship, how many FOVs had both mask types and contributed evidence;
  • whether the segmentation snapshot is partial or complete;
  • enough mask fingerprint/digest provenance to identify the exact mask snapshot used.

Use one clear policy in the notebook/CLI:

  • either scan the current segmentation snapshot and say so explicitly; or
  • require the exact masks recorded by the preprocessing run.

For the intended progressive-segmentation workflow, scanning the current snapshot with explicit provenance is probably the simpler and more useful model.

Acceptance

  • A report from 60/100 segmented FOVs cannot be mistaken for a complete 100-FOV report.
  • Missing masks are visible as completeness information, not only warnings outside the report.
  • Relationship evidence states the number of FOVs that actually contributed.
  • A mask added/changed after the preprocessing run is explicitly identifiable in the scan provenance.
  • The notebook wording matches the implemented snapshot policy.
  • A final scan after segmentation completion is unambiguously marked complete.

Relevant code: mask_scan.MaskReport, run/mask_step.py, notebook mask-step text.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions