Skip to content

Fail on output filename collisions before processing - #4974

Merged
daniellerozenblit merged 2 commits into
meta:devfrom
daniellerozenblit:fix-output-collisions
Nov 18, 2026
Merged

daniellerozenblit merged 2 commits into
meta:devfrom
daniellerozenblit:fix-output-collisions

Conversation

@daniellerozenblit

Copy link
Copy Markdown
Contributor

With --output-dir-flat, inputs from different directories can map to the same output file issue #4854 Previously, zstd only warned about it, and only after all files were processed, so with -f the earlier outputs were silently overwritten, and with --rm their inputs were deleted too.

This PR changes behavior so that we now:

  • Check for collisions before processing any file, and fail the operation if any are found.
  • For decompression, check the computed output names rather than the input names: different suffixes can map to the same output, e.g. a.tar.zst and a.tzst both decompress to a.tar.

With --output-dir-flat, inputs from different directories can map to
the same output file. zstd only warned about it, and only after all
files were processed, so with -f the earlier outputs were silently
overwritten, and with --rm their inputs were deleted too.

- Check for collisions before processing any file, and fail the
  operation if any are found.
- For decompression, check the computed output names rather than the
  input names: different suffixes can map to the same output, e.g.
  a.tar.zst and a.tzst both decompress to a.tar.
- Skip the check with --output-dir-mirror, which keeps outputs in
  separate directories.
- Update the man page, which described the old resolution rules.
@meta-cla meta-cla Bot added the CLA Signed label Nov 16, 2026
@daniellerozenblit

Copy link
Copy Markdown
Contributor Author

Will add the fix for windows in a separate PR.

@Cyan4973 Cyan4973 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With --exclude-compressed, files that would be skipped still take part in the collision check: zstd --exclude-compressed --output-dir-flat out d1/x.zst d2/x.zst d1/y now fails without processing anything, while dev compresses d1/y. The check could skip the files that --exclude-compressed excludes.

Files that --exclude-compressed skips don't produce an output, so they
can't collide. Leave them out of the --output-dir-flat collision check
so that the other files are still processed.
@daniellerozenblit

Copy link
Copy Markdown
Contributor Author

@Cyan4973 Good catch, thanks! Files skipped by --exclude-compressed are now left out of the collision check. I added your example to the tests in output_dir_collision.sh.

@daniellerozenblit
daniellerozenblit merged commit 779f733 into meta:dev Nov 18, 2026
110 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants