Skip to content

Fix producer flush hangs when undrained batches expire - #3165

Open
zhechen-cs wants to merge 1 commit into
dpkp:masterfrom
zhechen-cs:fix-producer-expiry-unmute
Open

zhechen-cs wants to merge 1 commit into
dpkp:masterfrom
zhechen-cs:fix-producer-expiry-unmute

Conversation

@zhechen-cs

@zhechen-cs zhechen-cs commented Oct 11, 2026 •

Copy link
Copy Markdown

Completing an undrained expired batch with message ordering enabled removes a partition that was never muted. The resulting KeyError interrupts expiry processing, leaving other batches' futures unresolved and causing flush() to wait indefinitely.

Use discard() when unmuting partitions in both completion paths. Regression tests cover multiple undrained expired batches, late success/error responses after expiry, and normal successful completion with message ordering enabled or disabled.

Fixes #3159.

Validation:

  • The three regression cases fail with KeyError on unmodified master and pass with the fix.
  • pytest -q test --ignore=test/integration: 2743 passed, 1 skipped (Python 3.12.13).
  • make lint and git diff --check passed.

Kafka broker integration tests were not run locally.

CI note: all 13 failing matrix jobs report test/net/backend/test_transport.py::TestKafkaTCPTransport::test_ssl_wrapper_transport raising ValueError: check_hostname requires server_hostname. The same test fails on clean base cd3f7938 and PR head 70d9c08e with Python 3.13.16 and the same development dependencies. This failure is independent of the producer expiry change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sender: KeyError from muted.remove() while expiring batches orphans the rest; flush() hangs forever

1 participant