Repository navigation
decompress: reset streaming session in one-shot APIs, document prefix lifetime - #4975
Merged
Merged
Conversation
Contributor
Author
|
This leaves the current behavior of |
terrelln
force-pushed
the
2026-11-16-prefix-lifetime
branch
from
November 16, 2026 23:43
33d9523 to
dc2f685
Compare
terrelln
force-pushed
the
2026-11-16-prefix-lifetime
branch
2 times, most recently
from
November 17, 2026 19:03
6c0af7e to
582f8fb
Compare
Contributor
Author
|
I've simplified the tests, but they are still quite long because they're adding tests for the entire contract around when the ref-prefix is dropped, and how single-shot compression interacts with streaming. |
Cyan4973
approved these changes
Nov 18, 2026
Cyan4973
left a comment
Contributor
There was a problem hiding this comment.
Good: a one-shot call now abandons any stream in progress on the same DCtx, which feels more logical.
The updated behavior also matches what zstd.h already promised.
… lifetime ZSTD_decompressDCtx(), ZSTD_decompress_usingDict() and ZSTD_decompress_usingDDict() now begin with the equivalent of ZSTD_DCtx_reset(dctx, ZSTD_reset_session_only). The documentation already claimed that these functions reset a DCtx left in an undefined state by a streaming error, but they did not. As a result: - A DCtx left mid-stream or in error stayed dirty after a successful one-shot call, so the next streaming frame was misparsed. - ZSTD_decompressDCtx() could consume a prefix left over from a stream that was started and abandoned. Sticky dictionaries and parameters are unaffected, and an unused prefix still survives until a one-shot call uses it. The reset is a few stores and a branch per call. The streaming single-pass shortcut called ZSTD_decompress_usingDDict() internally. It now calls ZSTD_decompressMultiFrame() directly, so it does not reset the stream it is running in. Clarify one-shot and streaming prefix consumption, reset behavior, and error recovery in zstd.h. Add tests covering frame boundaries, skippable frames, explicit dictionaries, reset directives, prefix replacement, and one-shot calls on a DCtx in each dirty streaming state (including with an abandoned prefix and with multiple DDicts).
terrelln
force-pushed
the
2026-11-16-prefix-lifetime
branch
from
November 18, 2026 16:31
582f8fb to
8c829d3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ZSTD_decompressDCtx(), ZSTD_decompress_usingDict() and
ZSTD_decompress_usingDDict() now begin with the equivalent of
ZSTD_DCtx_reset(dctx, ZSTD_reset_session_only). The documentation already
claimed that these functions reset a DCtx left in an undefined state by a
streaming error, but they did not. As a result:
one-shot call, so the next streaming frame was misparsed.
was started and abandoned.
Sticky dictionaries and parameters are unaffected, and an unused prefix still
survives until a one-shot call uses it. The reset is a few stores and a
branch per call.
The streaming single-pass shortcut called ZSTD_decompress_usingDDict()
internally. It now calls ZSTD_decompressMultiFrame() directly, so it does not
reset the stream it is running in.
Clarify one-shot and streaming prefix consumption, reset behavior, and error
recovery in zstd.h. Add tests covering frame boundaries, skippable frames,
explicit dictionaries, reset directives, prefix replacement, and one-shot
calls on a DCtx in each dirty streaming state (including with an abandoned
prefix and with multiple DDicts).