Repository navigation
Turn content hashing of the sources on by default - #189
Merged
Merged
Conversation
Without it an image edited in place under the same size and modification time was reused as unchanged. A run now records each image's content SHA-256 unless it opts out (RunMode(verify_content=False), --no-verify-content, the notebook's box unticked). The manifest always records the choice (settings.verify_content, true or false). A first run hashes; an existing output folder keeps the choice of its earlier run. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm # Conflicts: # CHANGELOG.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Scma6W3LC11s3qjXHZuGDm # Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
Fixes #159 as decided by the owner (2026-11-07): content hashing ON by default, the cost benchmarked on a realistic folder, and an explicit opt-out recorded in the manifest.
Without content hashing, an image rewritten in place with its size and modification time kept counted as unchanged and its earlier rows were reused.
verify_contentchoice (RunMode()default,run_experimentdefault, CLI without a flag) records each image's content SHA-256 and writes manifest version 3. The notebook's "Verify each image's content" box is ticked by default (and set back to ticked, not unticked, when filled values are reset).RunMode(name, verify_content=False),--no-verify-content, or unticking the box. The manifest records it:settings.verify_contentis now written by every version 2 and 3 manifest (trueorfalse). Version 1 manifests are unchanged. A manifest written before this change has no key and still reads as not verified.--verify-content(this recomputes every row, as turning it on always did). A first run, or a rebuild from a version 1 manifest, hashes.maskscommand is unchanged; its default still follows the recorded run.docs/contract.md,docs/user_guide.md,CHANGELOG.md, CLI help.Benchmark
60 OME-TIFF images (2 conditions x 3 repeats x 10; 3 channels, 2048 x 2048, uint16; 25.2 MB each, 1.51 GB), generated locally, 4 vCPU, page cache dropped before every timed run, 3 repetitions (
run_experiment, confirmed design, restart mode):Raw SHA-256 of the same files, cold: 1.43 s (about 1,060 MB/s). Reading about 1.5 GB takes roughly 1.5 to 4 s extra, so the cost is about 2 to 3 seconds per GB of sources on this machine. It is not a bound for slower storage: on a network drive the extra reads would take longer. The benchmark script is not part of the PR.
Tests
New
tests/test_content_hash_default.py, all 4 failing on the old code and passing now:test_a_run_given_no_mode_hashes_every_source_and_records_ittest_a_default_run_reads_an_image_rewritten_with_its_metadata_kepttest_an_opt_out_is_recorded_as_false_and_the_rewrite_is_not_seentest_the_cli_hashes_by_default_and_no_verify_content_opts_outExisting tests that relied on the old default now opt out explicitly (version 1 and 2 manifest fixtures, "default run does not see a rewrite" blind-spot tests, the notebook box test, the byte-identity test against the pre-version-3 code, which now allows the one new key).
test_the_cli_turns_content_verification_onand similar already used a hashing run and are unchanged.Checks
ruff check .andruff format --check .: clean.python tools/check.py --rules: passes.tools/check.pyas a whole, and other Python versions.Downstream: bricks reading a run through
RunInputsalready read version 3 manifests (RunInputs.openchecks every SHA-256 once per open, which is now the default cost). This was not tested against MorphoFeature in this PR.Generated by Claude Code