Repository navigation
Intensity checks and plots from the histogram table (P5) - #29
Merged
Merged
Conversation
…ook (P5) After every written run, and in step 4 for any table, the normalization brick's image_summaries and intensity_checks give the four intensity warnings from the table alone (intensity module). The notebook shows them in steps 3 and 4 with a collapsed Check settings panel and per-channel box plots and cumulative distributions per biological repeat; nothing is written. The command line prints them, with --outlier-limit, --saturation-limit and --min-images; the manifest records the limits as the optional key intensity_checks (version 1). Plan question 30. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up P5 of the normalization plan (
docs/followup_preprocessing.md§P5 in morphobricks-normalization; its contract §7 and decisions N38, N50–N53): intensity checks and plots from the histogram table. Plan §12 question 30. Stacked on #28 (P4): its base is the P4 branch, so the diff shows P5 only; merge #28 first, then retarget this PR tomain.Summary
intensity(new):check_intensities(records, settings)runs the normalization brick'simage_summariesandintensity_checks(IntensityCheckSettings) and returns anIntensityReport(summaries, the brick'sIssues withchannel, the limits;flagged_images(channel),shifted_repeats(channel),validation_issues());check_table(path)reads any table first. For the plots:repeat_boxes(report, channel)(image medians and high percentiles per condition and biological repeat, flagged images, shifted repeats) andrepeat_distributions(records, channel)(pooled histogram per biological repeat within each condition).intensity_textfor the CLI. No image is read; nothing is written.run:run_experiment(..., intensity_settings=None)checks the rows it wrote, in memory;RunResult.intensity(None when nothing is written). The warnings never block and stay out ofRunResult.issues.manifest: optional keyintensity_checks=IntensityCheckSettings.to_dict()(manifest version 1). Changing the limits recomputes nothing.cli:runprints the checks after the summary, grouped by kind with the action once per kind;--outlier-limit,--saturation-limit(a fraction),--min-images; invalid limits exit 1 with the brick's message. No plots.notebook_ui:IntensityPanel, shown in step 3 after every successful run (run_checks, for the table just written) and in step 4 for the chosen table (table_checks): status with the limits, tiles per kind, a warnings table (check, channelC<i>, image or repeat, what was found) with what to do per kind, a collapsed Check settings accordion (outlier limit 3, saturated pixels 1 %, fewest images 5, Restore the defaults; changes update the warnings at once and step 3's next run records them), channel buttons, box plots (red diamonds = images with a warning in that channel fromIssue.channel; amber boxes = shifted repeats) and cumulative distributions per biological repeat, one panel per condition (shifted repeats dashed, ≤ 12 conditions). Step 3's "What next?" asks to read the checks when there are warnings.Decisions
Asked of the owner before coding: warnings and plots in both step 3 (automatically after the run) and step 4; the run's limits recorded in the manifest (changing them afterwards only updates the screen until the next run); cumulative curves for the distributions. From the spec: the brick's functions, never re-reading images; comparisons within a condition; plots in the notebook only; the CLI prints the warnings.
Notes for review
outlier_limitandmin_imagesare the brick's (min_images provisional there).embed_minimal_htmland looked at in a browser, and the plot PNGs inspected (standalone renders lack JupyterLab styles and icons). Not looked at in a real JupyterLab.Tests
ruff check .andruff format --check .clean; LF line endings.tests/test_intensity.py(the warnings equal the brick'simage_summaries+intensity_checkson the same rows; channels, flagged images, shifted repeats, limits, box and curve data, table reading); run (checks in memory equalcheck_tableon the written CSV, limits recorded, new limits read no image), manifest round trip, CLI (grouped output, conditions never compared, flags, invalid limits), notebook (shown after the run, no extra file written, limits update at once and are recorded by the next run, channel buttons, step 4 for a table whose images were deleted).🤖 Generated with Claude Code