Skip to content

Add RGB-to-scalar stain presets and the preset catalogue - #95

Merged
guijacquemet merged 12 commits into
mainfrom
ccr-52991b66-5vnbgy
Nov 13, 2026
Merged

guijacquemet merged 12 commits into
mainfrom
ccr-52991b66-5vnbgy

Conversation

@guijacquemet

@guijacquemet guijacquemet commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Refs #94

Summary

The P0 stain presets (rgb_to_scalar, H&E first, plan N67), plus the preset catalogue the owner approved in #94 (comment) (plan N68, contract §3.4).

  • to_scalar(pixels, *, axes, preset, white=None, stain_vectors=None, polarizer=None) -> ScalarResult is the general entry point. rgb_to_scalar calls it with no optional arguments. Outputs of he_eosin, he_hematoxylin, luminance_inverted and lab_a are bitwise identical to b152472: same SHA-256 on a 64x80 random image in YXC and CYX, and a regression test compares them with the direct scikit-image calls.
  • StainPreset gains modality, read-only consumer hints (polarity, scaling, mask, measures), needs, inverted, target_stain and exclusion_stain. The hints are recorded in the provenance and are not applied here. PRESETS_BY_MODALITY gives a read-only listing by modality.
  • custom_od is written here in NumPy, without scikit-image (optical_density.py):
    • OD = -log10(max(I, 1) / white), using a declared white in (0, 255].
    • Unit stain vectors. With two vectors, the third is their normalised cross product ("residual").
    • Vectors less than 5 degrees apart are refused, and so is a condition number above 50.
    • C = OD @ inv(M). Negative concentrations are kept, not clipped.
    • The provenance reports negative_fraction, above_white_fraction and floored_fraction.
  • Helpers: white_from_region (median over the background) and stain_vectors_from_regions (normalised mean OD per labelled region, as in TWOMBLI).
  • Arguments a preset does not use are refused, never ignored.
  • New pure modules optical_density.py and scalar_inputs.py are added to the layer map. Every file stays at or below 600 lines.
  • rgb.py, NormalizationParameters and plane_histograms.csv are unchanged.
preset modality method image exclusion needs scaling / mask hint
he_eosin brightfield fixed_deconvolution hed_from_rgb comp. 1 haematoxylin — fixed / exclude_nuclei_rbc_empty
he_hematoxylin brightfield fixed_deconvolution hed_from_rgb comp. 0 — — fixed / none
luminance_inverted brightfield luminance 1 - rgb2gray — — (white optional, N69) fixed / exclude_nuclei_rbc_empty
lab_a brightfield cielab a* — — (white optional, N69) fixed / exclude_nuclei_rbc_empty
fluorescence fluorescence passthrough YX plane as float32 — — per_image / optional
shg shg passthrough YX plane as float32 — — fixed / none
psr_brightfield brightfield custom_od psr counterstain white, stain_vectors fixed / exclude_counterstain
psr_polarized polarized luminance rgb2gray (not inverted) — polarizer (white optional, N69) fixed / background
trichrome_blue brightfield fixed_deconvolution bpx_from_rgb comp. 0 (Methyl Blue) — — fixed / exclude_nuclei
trichrome_green brightfield cielab −a* — — (white optional, N69) fixed / exclude_nuclei
he_custom brightfield custom_od eosin hematoxylin white (vectors default, N69) fixed / exclude_nuclei_rbc_empty
trichrome_custom brightfield custom_od collagen ponceau_fuchsin (was nuclei, N69) white (vectors default, N69) fixed / exclude_nuclei

For psr_polarized, polarizer must be "circular" or "linear". The provenance hints record orientation_valid = (polarizer == "circular").

Tests

tests/test_stain.py covers P0 and the regression. tests/test_stain_catalogue.py covers the catalogue. tests/test_optical_density.py covers the OD maths. tests/stain_simulation.py makes the synthetic Beer–Lambert images.

  • Beer–Lambert round trip. The test image is I = white * 10^(-C @ M) with white (240, 235, 245).
    • Float input recovers C to 1e-12, with 3 explicit vectors and with 2 vectors plus the cross product.
    • uint8 input is checked against a per-pixel bound derived from rounding: sum_c log10(r_c/(r_c-0.5)) * |inv(M)[c,j]|. This runs for every custom_od preset, through to_scalar with CYX input.
  • Analytic checks: cond = cot(θ/2) for two vectors; scale invariance of the vectors.
  • Refusals:
    • stain vectors: missing or extra names, collinear or anti-parallel vectors, an angle of 4 degrees, nearly coplanar (ill-conditioned) vectors, non-finite values, zero length, wrong length or type;
    • white: out of range, wrong shape or type;
    • missing white or stain_vectors; missing or invalid polarizer; unused arguments;
    • passthrough: wrong axes or ndim, bool, complex, NaN, inf, float32 overflow.
  • Reported fractions: negative, above-white and floored fractions on constructed pixels (negative concentrations are not clipped). The declared white is used rather than 255.
  • Helpers: white_from_region returns the background median. stain_vectors_from_regions recovers the generating vectors (angle under 0.2–0.5 degrees), and unmixing with them separates the regions.
  • Other presets:
    • psr_polarized equals rgb2gray;
    • trichrome_blue equals separate_stains(bpx)[..., 0], and zero_fraction is reported;
    • trichrome_green equals -rgb2lab[..., 1].
  • Passthrough keeps the values, returns float32, does not alias the input, and reports whether the cast was exact (float32_exact).
  • Shared contract: YXC/CYX bitwise identical for all ten RGB presets; no input mutation; provenance and hints read-only; registry contents and order.
  • Without scikit-image: a subprocess test runs passthrough and custom_od with skimage blocked, and checks that the other presets raise ImportError.

Check results (at 30bab3d, which has the same code as beadf62 plus docs and CI)

  • python tools/check.py in a venv with .[dev,histology] (scikit-image 0.26.0): exit 0. 1955 passed, 2 skipped. The two skips are tests/test_rgb.py, because csbdeep is not installed.
  • In a venv with .[dev] only (no scikit-image), pytest tests/test_stain.py tests/test_stain_catalogue.py tests/test_optical_density.py: 162 passed, 41 skipped. Every skip is a test that needs scikit-image. Exit 0.
  • The histology CI job now also runs the two new test files.

Follow-up: default stain vectors and declared white (owner decisions of 13 November 2026, plan N69)

Implements #94 (comment) (commits 32c48d7, ee72fe3).

  • Default stain vectors, frozen read-only data in the preset record (default_stain_vectors, default_stain_vectors_citation), copied into the code (the presets do not need scikit-image):
    • he_custom: hematoxylin (0.65, 0.70, 0.29) and eosin (0.07, 0.99, 0.11), Ruifrok & Johnston 2001, rows 0 and 1 of skimage.color.rgb_from_hed.
    • trichrome_custom: collagen = Methyl Blue (0.7995107, 0.5913521, 0.10528667) and ponceau_fuchsin (0.09997159, 0.73738605, 0.6680326), rows 0 and 1 of rgb_from_bpx (Fiji's Masson Trichrome).
    • psr_brightfield: no default; stain_vectors stays required.
    • A given stain_vectors overrides the defaults. The provenance records stain_vectors_source ("preset_default" or "user") and stain_vectors_citation. white stays required for every custom_od preset.
  • Choice for trichrome: the exclusion stain of trichrome_custom is now ponceau_fuchsin (it was nuclei). The bpx matrix has no nuclear stain, so a default named nuclei would be wrong; returning Ponceau Fuchsin keeps the two-stain custom_od design without a special case. The mask hint stays exclude_nuclei: the nuclear mask is built in preprocessing, not taken from this map. User vectors for this preset now name collagen and ponceau_fuchsin (breaking for the draft only; preset version stays 1 since it is unreleased).
  • white for the colour projections: luminance_inverted, lab_a, psr_polarized and trichrome_green accept an optional white, checked like custom_od's. The RGB is rescaled to clip(rint(I * 255 / white), 0, 255) as uint8 (float64, rint half to even) before scikit-image. Provenance: white, white_applied, white_rescale, white_clipped_fraction (a channel above 255 after rounding) and above_white_fraction. Without white the outputs are bitwise unchanged, and white = (255, 255, 255) is the identity.
  • Not added to he_eosin, he_hematoxylin or trichrome_blue: separate_stains fixes white at 255, and he_custom and trichrome_custom are the deconvolution path for a declared white. These presets still refuse white. Passthrough presets refuse it too.
  • New StainPreset.accepts: arguments a preset uses when given but does not require. Everything outside needs | accepts is still refused.
  • Registry moved to the new pure module stain_presets.py, re-exported by stain.py (stain.py would have exceeded the 600-line rule).
  • Tests (new tests/test_stain_defaults.py, plus updates to test_stain_catalogue.py):
    • the defaults equal the scikit-image rows exactly (and rgb_from_bpx row 2 is their cross product);
    • default vs user provenance; the same numbers given as user vectors give bitwise the same maps;
    • Beer–Lambert round trip with the default vectors for he_custom and trichrome_custom, within the uint8 rounding bound;
    • the defaults work without scikit-image (subprocess);
    • the rescale formula on analytic values (1.5 → 2, 4.5 → 4, 300 → 255 clipped, 255.4 → 255 not clipped);
    • all four white presets equal scikit-image on the manually rescaled image, with the clipped fraction;
    • white 255 gives bitwise the outputs without white;
    • a background at the declared white becomes neutral;
    • invalid white refused; YXC/CYX identical with white.
    • The P0 parity test is unchanged and passes.
  • Docs: contract §3.4 (defaults table with citations, white option, provenance), plan N69, CHANGELOG. CI histology job runs test_stain_defaults.py.
  • Checks (at 32c48d7 / ee72fe3): python tools/check.py with .[dev,histology] (scikit-image 0.26.0, Python 3.11): exit 0, 2016 passed, 2 skipped (csbdeep). Without scikit-image, the four stain/OD test files: 204 passed, 60 skipped (every skip needs scikit-image).

Not in this PR

  • fixed_range stays None; it needs calibration on a reference set.
  • Colour normalization, 16-bit and ICC input, and hue for polarized PSR.
  • Default thresholds for masks on stain maps (owner decision: per-preset default where justified, otherwise user-set).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW

New stain module with rgb_to_scalar and a read-only registry of four P0
presets (he_eosin, he_hematoxylin, luminance_inverted, lab_a). 8-bit sRGB,
axes YXC or CYX, refusals instead of repairs, optional histology extra
(scikit-image, imported lazily). Contract section 3.4 and plan N67 record
the scope change; rgb.py and the file formats are untouched.

Refs #94

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW
The main CI matrix installs only .[dev], so tests/test_stain.py was skipped
there; a dedicated histology job installs the extra and runs it. _to_yxc
returned a view for contiguous YXC input (np.ascontiguousarray does not
copy); it now always copies, as its docstring says.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW
to_scalar(pixels, *, axes, preset, white, stain_vectors, polarizer) is the
general entry point; rgb_to_scalar delegates to it and the four P0 presets
give bitwise identical outputs. New presets: fluorescence and shg
(passthrough), psr_brightfield, he_custom and trichrome_custom (custom_od,
optical density against a declared white in NumPy), psr_polarized (rgb2gray,
declared polarizer), trichrome_blue (bpx_from_rgb) and trichrome_green
(minus CIELAB a*). Each preset carries a modality and consumer hints.
Helpers white_from_region and stain_vectors_from_regions derive the white
and the stain vectors from labelled regions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW
…ests in CI

Contract §3.4 gets the preset table with modality, method, target and
exclusion, hints, refusals, the optical-density definitions and limits.
Plan N68 records the owner's catalogue decision; CHANGELOG and the AGENTS.md
repository map list the new modules. The histology CI job now runs the
catalogue and optical-density tests too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW
@guijacquemet guijacquemet changed the title Add RGB-to-scalar stain presets (H&E first) Add RGB-to-scalar stain presets and the preset catalogue Nov 13, 2026
Python 3.11's dataclasses refuse a MappingProxyType default as mutable
(3.13 accepts it), so test collection failed on the 3.11 CI jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW
Review of b152472 (ChatGPT, verified against this head):
- np.ma.MaskedArray passed the ndarray check and np.array dropped its mask;
  masked arrays are now refused for RGB and plane presets.
- An image with Y or X of 0 reached np.mean and gave a NaN provenance;
  it is now refused.
- clipped_fraction counted every zero component, not only clipped ones;
  renamed zero_fraction (unreleased API).
- polarity 'bright' is a signal direction, not a stain amount.
Tests: masked and empty refusals, fixed reference values (white, black,
neutral grey). The pixelwise scikit-image parity test of the P0 presets
was already present since 5040a67's parent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW
…tions

he_custom and trichrome_custom get published default stain vectors
(Ruifrok-Johnston H&E rows of rgb_from_hed; Methyl Blue and Ponceau
Fuchsin rows of rgb_from_bpx), used when stain_vectors is not given and
recorded as stain_vectors_source = preset_default with a citation;
psr_brightfield keeps requiring user vectors. trichrome_custom's
exclusion stain is now ponceau_fuchsin (the published matrix has no
nuclear stain). white stays required for custom_od.

luminance_inverted, lab_a, psr_polarized and trichrome_green accept an
optional white: the RGB is rescaled to clip(rint(I * 255 / white), 0,
255) as uint8 before scikit-image, with the clipped fraction recorded.
Without white the outputs are bitwise unchanged; white = 255 is the
identity. The preset registry moves to stain_presets.py (file length).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW
… plan N69)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW
StainPreset.image_name (recorded in the provenance as image_name) names the image
a preset makes, as CellProfiler's UnmixColors names its outputs: he_eosin -> Eosin,
he_hematoxylin -> Hematoxylin, trichrome_blue -> MethylBlue, psr_brightfield ->
SiriusRed, and so on; unique tokens, which consumers use as the derived channel's
name in feature columns (owner decision: CellProfiler-style names). Contract §3.4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW
3a7af3e)

- check_white and stain vectors refuse a masked array, or a list holding a
  masked value, before np.asarray could drop the mask (an all-false mask too).
- Region helpers refuse a masked region array.
- stain_vectors_from_regions refuses a reference region holding a pixel with a
  channel at 0 (its OD would be taken at the floor and bend the vector); zeros
  outside the regions are allowed.
- MAX_CONDITION documented as a heuristic norm-wise threshold, not a per-map
  error guarantee; psr_polarized's green-weighted greyscale bias documented;
  white_from_region's median-255 refusal says a declared white of 255 is accepted.
- Regression tests through to_scalar and the public region helpers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW

Copy link
Copy Markdown
Contributor Author

External review of 3a7af3e: response (fixes in 8cc45da)

All findings still applied at the newer head and are fixed:

Finding Fix
B1: masked white or stain-vector components unmasked by np.asarray Refused (TypeError): a masked array, an all-false mask, or a list holding np.ma.masked. Tested through to_scalar
B2: masked boolean regions accepted Refused in both region helpers. Tested with one masked selected pixel next to valid ones
B3: reference regions with zero channels silently floored stain_vectors_from_regions refuses a region holding a pixel with a channel at 0. Zeros outside the regions are allowed. Both cases are tested
FILE 1: psr_polarized greyscale bias Contract §3.4 documents rgb2gray's weights (0.2125 R + 0.7154 G + 0.0721 B), which favour green, and their effect on fibres of different colours
FILE 2: condition-number claim MAX_CONDITION is now described as a heuristic norm-wise threshold, not a per-map or per-pixel error guarantee. The measured value stays in the provenance
C7 suggestion white_from_region's refusal of a median of 255 now says an unsaturated white cannot be inferred from that region, and that a declared white of 255 is accepted

The reviewer confirmed cond = cot(θ/2) for the acute angle, and that OD @ inv(M) is correct for stain vectors stored as rows. The Beer–Lambert caveat (scanner transfer function) stays documented.

Also new since the review: 734c808 adds StainPreset.image_name, a CellProfiler-style name per preset (he_eosin → Eosin), recorded in the provenance. Owner decision on column naming.

Full check at 8cc45da: 2023 passed, 2 skipped, exit 0.


Generated by Claude Code

… (review of 8cc45da)

- Contract §3.4: he_custom is deliberately not the three-stain HED conversion
  (cross-product residual instead of DAB, log10 OD against the declared white, no
  clipping); the wrong "completion of rgb_from_hed" sentence is corrected; the
  declared-white path's uint8 re-quantisation (at most half a grey level) is
  documented.
- Tests: literal published vectors independent of scikit-image; he_custom on a
  reference pixel computed by hand; its difference from scikit-image's HED eosin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW

Copy link
Copy Markdown
Contributor Author

External review of 3a7af3e→8cc45da (verdict: RELEASE AFTER FIXES): response, fixed in 2506e51

  • Blocker (a claimed he_custom = HED equivalence): he_custom is deliberately a two-stain model. It has Ruifrok & Johnston's haematoxylin and eosin vectors, a cross-product residual instead of DAB, log10 optical density against the declared white, and no clipping. Contract §3.4 now says so. The equivalence had been claimed only in my review prompt and in one wrong contract sentence.
    • New tests: the literal published vectors, checked independently of scikit-image.
    • he_custom on a reference pixel, computed by hand from the literal unit matrix. Eosin is 0.294 here, while scikit-image's HED eosin is 0.046 for the same pixel.
    • A test that the two models differ.
    • Use he_eosin for the published HED conversion.
  • FILE 1: the cross-product sentence is corrected. rgb_from_bpx is completed by the cross product; rgb_from_hed has an explicit DAB row.
  • FILE 2: the declared-white path's uint8 re-quantisation is documented: at most half a grey level per channel, plus the reported clipping. Float input would avoid it; the method stays fixed as stated.
  • FILE 3: the literal reference table is added, alongside the existing comparison with scikit-image.
  • The reviewer confirmed: the default vectors match their published sources (the paper, scikit-image 0.26 and Landini's Masson Trichrome); the optional-white path is correct; the three earlier blockers are fixed.

Full check at 2506e51: 2026 passed, 2 skipped, exit 0. Ready for the owner's review and tag.


Generated by Claude Code

pyproject 0.6.0; CHANGELOG 'Unreleased' becomes v0.6.0 (2026-11-13), an
additive release (stain presets, optional extra histology; nothing that existed
changes) with the known limitations the release review asked to state; README
release line, install command and a stain-presets bullet; plan status;
AGENTS.md decision range N1-N69 (release review FILE 1). No code change.
Checked: full check 2026 passed; a clean NumPy-only install reports 0.6.0, runs
he_custom and refuses he_eosin naming the histology extra; a clean
[histology] install runs he_eosin (scikit-image 0.26.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VbytvF3Gsa85ix8Ne2yMDW

Copy link
Copy Markdown
Contributor Author

Release review (prompt I, at 2506e51): RELEASE AFTER FIXES. Fixed in a5ee164

The release review confirmed all earlier findings as fixed and CI green on all 10 jobs at 2506e51. It found no defect in the numerical code and no change to legacy code. Its two remaining points:

  • B1, release identity: pyproject.toml said 0.5.0. It is now 0.6.0. The CHANGELOG Unreleased heading becomes v0.6.0 (2026-11-13), with the known limitations the reviewer listed:

    • 8-bit sRGB only;
    • he_custom is not HED;
    • zero_fraction cannot tell zeros from clipped negatives;
    • uint8 re-quantisation with a declared white;
    • one Masson profile;
    • linear-polarizer bias;
    • not collagen-specific;
    • Beer–Lambert is an approximation on scanner sRGB;
    • scaling hints are never applied.

    The README release line and install command now point to @v0.6.0, and the plan status says v0.6.0.

  • FILE 1: AGENTS.md now gives the decision range as N1–N69.

a5ee164 changes metadata and docs only (5 files, no code). Checks:

  • full check: 2026 passed, 2 skipped;
  • a clean NumPy-only install reports 0.6.0, runs he_custom, and refuses he_eosin with a message naming the histology extra;
  • a clean [histology] install runs he_eosin (scikit-image 0.26.0).

To release: wait for CI on a5ee164, merge, and tag v0.6.0 on the merge commit. Then preprocessing, CellPatcher and fiberfeatures move their normalization pin to v0.6.0 together.


Generated by Claude Code

@guijacquemet
guijacquemet marked this pull request as ready for review November 13, 2026 19:32
@guijacquemet
guijacquemet merged commit 95e1d6d into main Nov 13, 2026
10 checks passed
@guijacquemet
guijacquemet deleted the ccr-52991b66-5vnbgy branch November 13, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants