Skip to content

Intensity checks and plots from the histogram table (P5) - #29

Merged
IvanHCenalmor merged 1 commit into
mainfrom
claude/p5-intensity-checks-648ec4
Oct 27, 2026
Merged

IvanHCenalmor merged 1 commit into
mainfrom
claude/p5-intensity-checks-648ec4

Conversation

@IvanHCenalmor

Copy link
Copy Markdown
Contributor

Follow-up P5 of the normalization plan (docs/followup_preprocessing.md §P5 in morphobricks-normalization; its contract §7 and decisions N38, N50–N53): intensity checks and plots from the histogram table. Plan §12 question 30. Stacked on #28 (P4): its base is the P4 branch, so the diff shows P5 only; merge #28 first, then retarget this PR to main.

Summary

  • intensity (new): check_intensities(records, settings) runs the normalization brick's image_summaries and intensity_checks (IntensityCheckSettings) and returns an IntensityReport (summaries, the brick's Issues with channel, the limits; flagged_images(channel), shifted_repeats(channel), validation_issues()); check_table(path) reads any table first. For the plots: repeat_boxes(report, channel) (image medians and high percentiles per condition and biological repeat, flagged images, shifted repeats) and repeat_distributions(records, channel) (pooled histogram per biological repeat within each condition). intensity_text for the CLI. No image is read; nothing is written.
  • run: run_experiment(..., intensity_settings=None) checks the rows it wrote, in memory; RunResult.intensity (None when nothing is written). The warnings never block and stay out of RunResult.issues.
  • manifest: optional key intensity_checks = IntensityCheckSettings.to_dict() (manifest version 1). Changing the limits recomputes nothing.
  • cli: run prints the checks after the summary, grouped by kind with the action once per kind; --outlier-limit, --saturation-limit (a fraction), --min-images; invalid limits exit 1 with the brick's message. No plots.
  • notebook_ui: IntensityPanel, shown in step 3 after every successful run (run_checks, for the table just written) and in step 4 for the chosen table (table_checks): status with the limits, tiles per kind, a warnings table (check, channel C<i>, image or repeat, what was found) with what to do per kind, a collapsed Check settings accordion (outlier limit 3, saturated pixels 1 %, fewest images 5, Restore the defaults; changes update the warnings at once and step 3's next run records them), channel buttons, box plots (red diamonds = images with a warning in that channel from Issue.channel; amber boxes = shifted repeats) and cumulative distributions per biological repeat, one panel per condition (shifted repeats dashed, ≤ 12 conditions). Step 3's "What next?" asks to read the checks when there are warnings.
  • Notebook file: what the notebook does, quick workflow, step 3 and 4 text, a common problem quoting the outlier message.
  • Docs: contract §8, §9 (manifest key), new §14; user guide §4–§9 (with the four codes); README; plan §10 and §12 question 30; AGENTS.md.

Decisions

Asked of the owner before coding: warnings and plots in both step 3 (automatically after the run) and step 4; the run's limits recorded in the manifest (changing them afterwards only updates the screen until the next run); cumulative curves for the distributions. From the spec: the brick's functions, never re-reading images; comparisons within a condition; plots in the notebook only; the CLI prints the warnings.

Notes for review

  • Step 3's panel reads the written table once more to draw the plots (step 4 does the same); the run itself checks the rows in memory for the CLI and the manifest. Both give the same warnings (tested).
  • outlier_limit and min_images are the brick's (min_images provisional there).
  • Widgets tested headless; step 3 and 4 rendered with embed_minimal_html and looked at in a browser, and the plot PNGs inspected (standalone renders lack JupyterLab styles and icons). Not looked at in a real JupyterLab.
  • Nothing is tagged; the owner tags v0.2.0 after P4 and P5.

Tests

  • Locally (Windows 11, Python 3.12, zarr 3.4.2, notebook and zarr extras): 983 passed, 2 skipped (POSIX-only file-system tests); the notebook tests re-run after the last cosmetic plot edits (101 passed); ruff check . and ruff format --check . clean; LF line endings.
  • New: tests/test_intensity.py (the warnings equal the brick's image_summaries + intensity_checks on the same rows; channels, flagged images, shifted repeats, limits, box and curve data, table reading); run (checks in memory equal check_table on the written CSV, limits recorded, new limits read no image), manifest round trip, CLI (grouped output, conditions never compared, flags, invalid limits), notebook (shown after the run, no extra file written, limits update at once and are recorded by the next run, channel buttons, step 4 for a table whose images were deleted).
  • CI: GitHub Actions on this PR.

🤖 Generated with Claude Code

…ook (P5)

After every written run, and in step 4 for any table, the normalization
brick's image_summaries and intensity_checks give the four intensity
warnings from the table alone (intensity module). The notebook shows them
in steps 3 and 4 with a collapsed Check settings panel and per-channel box
plots and cumulative distributions per biological repeat; nothing is
written. The command line prints them, with --outlier-limit,
--saturation-limit and --min-images; the manifest records the limits as the
optional key intensity_checks (version 1). Plan question 30.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from claude/p4-image-type-648ec4 to main October 27, 2026 18:39
@IvanHCenalmor
IvanHCenalmor merged commit 27f04ef into main Oct 27, 2026
7 checks passed
@guijacquemet
guijacquemet deleted the claude/p5-intensity-checks-648ec4 branch November 1, 2026 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant