Repository navigation
Record failed image reads and mask failures in chunk_status - #3
Merged
Merged
Conversation
write_ncdf() records a per-chunk status, but only a band image that fails to OPEN was ever recorded as non-OK. Two other failures produced a chunk computed from partial input with status OK: - gdalwarp_client::warp_simple() ignored the return of ChunkAndWarpImage(), and warp_complex() only logged it at DEBUG, so the INIT_DEST=nan output came back as if it were data. A band whose tile read fails after a successful open (corrupted tile, refused or throttled HTTP range request) was silently dropped from the chunk; for a MASK band the scene was used unmasked, since a NaN mask value never matches a mask class. warp() now warns and returns nullptr on a failed Initialize() or ChunkAndWarpImage(), after the existing cleanup, so read_chunk()'s existing `!gdal_out` branches mark the chunk INCOMPLETE (ERROR when strict). - A mask image that fails to open `continue`d without set_status(); it now sets INCOMPLETE, and the strict branch returns an ERROR chunk like the band path does. The test builds a two-date collection of single-band COGs (a band and an SCL-style mask, the second date a cloud scene) from dummy cubes and breaks the second date three ways: a band file whose first tile is corrupted with the IFD intact, a deleted mask file, and a corrupted mask file. At parallel 1 and 2 each must give a non-OK status and never let the cloud value through; strict mode with a missing mask must give ERROR. Clean controls (same CRS, another CRS, partial coverage, a coarser view read through overviews) must stay OK. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRhUJsuKABLfpBGktPoiBN
This was referenced Oct 14, 2026
Closed
Owner
Author
|
Posted upstream as appelmar#114 (2026-10-14). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second of three independent PRs (see ).
chunk_statusrecords a band image that fails to open, but two other failures leave the chunk OK:/vsicurl/):warp_simple()ignoresChunkAndWarpImage()'s return, so the NaN output is used as data. For a mask image the scene is then used unmasked, since NaN never matches a mask value, and cloud reaches the composite.continues withoutset_status(), so the scene is dropped with status OK.Reproducible example (no network)
gdalcubes 0.7.5 · R 4.5.2 · macOS 15.6.1 (arm64) · GDAL 3.13.0. Two dates, a band and an SCL-style mask; the second date is all cloud, and its mask file is cut short so it opens but cannot be read.
On 0.7.5 the cloud scene is used unmasked (the median of 500 and 900) and the chunk is OK:
With this change:
500in all 1600 cells andchunk_status2 (incomplete).Change
warp()returnsnullptrwhenInitialize()orChunkAndWarpImage()fails, soread_chunk()'s existing!gdal_outbranches mark the chunk INCOMPLETE (ERROR whenincomplete_ok = FALSE), for band and mask images alike.One behaviour change to be aware of: an image for which GDAL cannot compute a source window also makes
ChunkAndWarpImage()fail, so it now gives INCOMPLETE instead of silent NaN with status OK.Adds
inst/tinytest/test_chunk_status.R(band read, mask open and mask read failures, plus clean controls; 10 of its 32 assertions fail on 0.7.5). The full suite passes. R-CMD-check onmasteris currently failing as well, independent of this change.Fork review copy. Everything above the line is the draft for the upstream PR to appelmar/gdalcubes; not posted. Replaces the issue drafted in #6, which is folded in here.
Relates to NewGraphEnvironment/drift#99
Relates to NewGraphEnvironment/sred-2025-2026#16
🤖 Generated with Claude Code