Skip to content

decompress: reset streaming session in one-shot APIs, document prefix lifetime - #4975

Merged
terrelln merged 1 commit into
meta:devfrom
terrelln:2026-11-16-prefix-lifetime
Nov 18, 2026
Merged

terrelln merged 1 commit into
meta:devfrom
terrelln:2026-11-16-prefix-lifetime

Conversation

@terrelln

@terrelln terrelln commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

ZSTD_decompressDCtx(), ZSTD_decompress_usingDict() and
ZSTD_decompress_usingDDict() now begin with the equivalent of
ZSTD_DCtx_reset(dctx, ZSTD_reset_session_only). The documentation already
claimed that these functions reset a DCtx left in an undefined state by a
streaming error, but they did not. As a result:

  • A DCtx left mid-stream or in error stayed dirty after a successful
    one-shot call, so the next streaming frame was misparsed.
  • ZSTD_decompressDCtx() could consume a prefix left over from a stream that
    was started and abandoned.

Sticky dictionaries and parameters are unaffected, and an unused prefix still
survives until a one-shot call uses it. The reset is a few stores and a
branch per call.

The streaming single-pass shortcut called ZSTD_decompress_usingDDict()
internally. It now calls ZSTD_decompressMultiFrame() directly, so it does not
reset the stream it is running in.

Clarify one-shot and streaming prefix consumption, reset behavior, and error
recovery in zstd.h. Add tests covering frame boundaries, skippable frames,
explicit dictionaries, reset directives, prefix replacement, and one-shot
calls on a DCtx in each dirty streaming state (including with an abandoned
prefix and with multiple DDicts).

@meta-cla meta-cla Bot added the CLA Signed label Nov 16, 2026
@terrelln

Copy link
Copy Markdown
Contributor Author

This leaves the current behavior of ZSTD_decompressDCtx() as-is. Instead, as discussed with @Cyan4973, I will update ZSTD_decompressDCtx() and other one-shot APIs to call ZSTD_DCtx_reset() so the state of a streaming decompression is reset.

@terrelln
terrelln force-pushed the 2026-11-16-prefix-lifetime branch from 33d9523 to dc2f685 Compare November 16, 2026 23:43
@terrelln terrelln changed the title decompress: document and test prefix lifetime decompress: reset streaming session in one-shot APIs, document prefix lifetime Nov 16, 2026
@terrelln
terrelln force-pushed the 2026-11-16-prefix-lifetime branch 2 times, most recently from 6c0af7e to 582f8fb Compare November 17, 2026 19:03
@terrelln

Copy link
Copy Markdown
Contributor Author

I've simplified the tests, but they are still quite long because they're adding tests for the entire contract around when the ref-prefix is dropped, and how single-shot compression interacts with streaming.

@Cyan4973 Cyan4973 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good: a one-shot call now abandons any stream in progress on the same DCtx, which feels more logical.
The updated behavior also matches what zstd.h already promised.

… lifetime

ZSTD_decompressDCtx(), ZSTD_decompress_usingDict() and
ZSTD_decompress_usingDDict() now begin with the equivalent of
ZSTD_DCtx_reset(dctx, ZSTD_reset_session_only). The documentation already
claimed that these functions reset a DCtx left in an undefined state by a
streaming error, but they did not. As a result:

- A DCtx left mid-stream or in error stayed dirty after a successful
  one-shot call, so the next streaming frame was misparsed.
- ZSTD_decompressDCtx() could consume a prefix left over from a stream that
  was started and abandoned.

Sticky dictionaries and parameters are unaffected, and an unused prefix still
survives until a one-shot call uses it. The reset is a few stores and a
branch per call.

The streaming single-pass shortcut called ZSTD_decompress_usingDDict()
internally. It now calls ZSTD_decompressMultiFrame() directly, so it does not
reset the stream it is running in.

Clarify one-shot and streaming prefix consumption, reset behavior, and error
recovery in zstd.h. Add tests covering frame boundaries, skippable frames,
explicit dictionaries, reset directives, prefix replacement, and one-shot
calls on a DCtx in each dirty streaming state (including with an abandoned
prefix and with multiple DDicts).
@terrelln
terrelln force-pushed the 2026-11-16-prefix-lifetime branch from 582f8fb to 8c829d3 Compare November 18, 2026 16:31
@terrelln
terrelln merged commit ae07916 into meta:dev Nov 18, 2026
110 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants