Skip to content

Add explicit channel-aware canonical dataset export to OME-TIFF or OME-Zarr #171

Open

Description

@guijacquemet

Goal

Allow a heterogeneous acquisition to be copied into a canonical MorphoBricks-ready dataset when biological repeats were acquired with different channel counts or channel orders.

Examples:

  • repeat 1: [DAPI, ACTIN, PAXILLIN]
  • repeat 2: [ACTIN, DAPI, PAXILLIN]
  • repeat 3: [DAPI, ACTIN, PAXILLIN, unused]

The user should be able to explicitly map these to one common output schema, e.g.
[DAPI, ACTIN, PAXILLIN], with the fourth channel of repeat 3 deliberately dropped.

Existing code to reuse

The current convert path is deliberately narrower:

  • it writes named-axis OME-TIFF copies;
  • it corrects/records axis interpretation;
  • it preserves pixel values and existing channel order;
  • it has strong source fingerprinting, planning, space checks, conversion records, crash-safe writes and reuse.

Do not bolt silent channel manipulation into the normal preprocessing run. Extend/reuse the conversion infrastructure or introduce a clearly named canonicalization operation with the same safety guarantees.

Required plan

Before writing anything, show a table per condition / biological repeat (or the narrowest required scope):

repeat      source channels              output channels             action
rep1        0:DAPI 1:ACTIN 2:PAXILLIN    DAPI ACTIN PAXILLIN         unchanged
rep2        0:ACTIN 1:DAPI 2:PAXILLIN    DAPI ACTIN PAXILLIN         reorder 1,0,2
rep3        0:DAPI 1:ACTIN 2:PAX 3:XYZ   DAPI ACTIN PAXILLIN         drop 3

Every reorder/drop must be explicit and user-confirmed.

Never:

  • infer a reorder from image appearance and apply it automatically;
  • fill a missing channel;
  • duplicate a channel to make counts match;
  • change pixel values silently.

If a required canonical channel is genuinely missing, refuse that mapping unless a future design explicitly supports missing-channel datasets.

Output formats

Support a canonical copy in one or both standard forms:

  • OME-TIFF;
  • OME-Zarr / OME-NGFF.

The format choice is operational; the canonical channel identity/order must be identical.

OME-Zarr export was previously removed from the old conversion path. If reintroduced here, do it as an explicit canonical-export feature with tests and provenance, not as a hidden preprocessing optimization.

Provenance

For every output image record:

  • source fingerprint/content identity;
  • source axes and channel names;
  • canonical output axes;
  • source-channel -> output-channel mapping;
  • dropped source channels;
  • canonical names;
  • output format;
  • output fingerprint/hash;
  • reason/scope for a non-identity mapping.

The copy tree must remain outside the source experiment and must never modify originals.

If OME-Zarr source label images/masks exist, their treatment must be explicit: preserve/copy them correctly or refuse with an actionable message. Never silently lose masks.

Notebook

Offer this only when the folder/channel checks reveal a correctable heterogeneous layout, or under an advanced Create canonical copy card.

The user should be able to:

  1. choose the canonical channels;
  2. define mappings for affected biological repeats;
  3. preview the plan and estimated disk use;
  4. choose OME-TIFF or OME-Zarr where supported;
  5. write the copy;
  6. switch preprocessing to the canonical copy and re-check it.

Acceptance

  • Reorder and drop operations are explicit and previewed before writing.
  • Output datasets have one canonical channel count/order/name schema.
  • Unchanged images may be reused/copied efficiently but provenance remains complete.
  • No missing channel is filled or guessed.
  • Original images are never changed.
  • Interrupted conversion is crash-safe and resumable/reusable using existing conversion principles.
  • A conversion record fully describes every channel transformation.
  • OME-TIFF and OME-Zarr outputs, when both supported, carry equivalent canonical metadata.
  • Final canonical dataset passes preprocessing's normal channel-count/name/order checks.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions