You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Conversion reopens TIFF headers for pixel reads and memory checks (performance follow-up of #96) #141
#96 is closed. The consistency problem it raised is fixed: the conversion no longer mixes metadata from different points in time, and it checks each source's fingerprint before and after it writes the copy (conversion_write._write_partial). The conversion still opens each TIFF header again more than once, and nobody has measured what that costs. The external review of v0.4.0rc4 found this. This issue tracks it as a performance item only.
What still reopens a header (main e24f334)
conversion_plan._cannot_convert → planes.check_planes(root, mappings, memory_limit=...) opens every TIFF to be converted, to check that its planes fit the memory limit. Preflight has already inspected those headers.
ome_tiff_copy (iter_planes(source, mapping, memory_limit=...), line 126) opens the source again to read the pixels. ome_tiff_copy line 134 opens a tifffile.TiffFile again for the copy's layout.
So one converted file costs several header opens, IFD reads and OME-XML parses on top of the preflight inspection.
What to do
Benchmark first. Count TiffFile opens and measure elapsed time and peak memory for 100, 1k and 10k TIFFs: small ones, files with many pages, files with heavy OME-XML, on a local disk and on a network or Drive mount. Compare the output byte for byte: copies, NGFF axis scales and units.
Activity