Repository navigation
subprocess.communicate never returns when the subprocess outputs more than 0xFFFF + 1 bytes #363
Description
- uvloop version: uvloop==0.14.0
- Python version: Python 3.8.6
- Platform: MacOS
- Can you reproduce the bug with
PYTHONASYNCIODEBUGin env?: yes - Does uvloop behave differently from vanilla asyncio? How?: yes. vanilla asyncio just works
The following code never returns with uvloop :
import asyncio
from asyncio.subprocess import PIPE
import uvloop
import logging
logging.basicConfig(level=logging.DEBUG)
async def main():
proc = await asyncio.create_subprocess_exec("/bin/cat", stdin=PIPE, stderr=PIPE, stdout=PIPE)
stdin = b'x' * 65_537
stdout, stderr = await proc.communicate(stdin)
print(stdout)
uvloop.install()
asyncio.run(main())changing 65_537 to 65_536 makes the script work as expected and returns immediately.
The debug output is :
% PYTHONASYNCIODEBUG=1 python xxx.py
DEBUG:asyncio:<Process 13746> communicate: feed stdin (65537 bytes)
DEBUG:asyncio:<Process 13746> communicate: close stdin
DEBUG:asyncio:<Process 13746> communicate: read stdout
DEBUG:asyncio:<Process 13746> communicate: read stderr
Activity
lovasoa commentedon Oct 13, 2020
I can reproduce the issue with the latest version of uvloop published on pypi (v0.14.0, which is almost one year old), but I cannot reproduce it with the latest development version ( 6ef69a7 ). It looks like the issue has been fixed by some other change. It would be nice if a new version could be published on pypi soon.
Add a regression test for large stderr reading
Add a regression test for large stderr reading
Add a regression test for large stderr reading
Add a regression test for large stderr reading
Add a regression test for large stderr reading
fantix commentedon Feb 5, 2021
Closed by #364. Please feel free to reopen if the issue persists.
lovasoa commentedon Feb 5, 2021
@fantix : The issue was already fixed in master when it was opened. The problem is that the latest released version is old and contains the bug: https://pypi.org/project/uvloop/#history
fantix commentedon Feb 5, 2021
Right, thanks for the reminder - this links to how we close issues. I'm leaning towards marking issues closed to reflect the situation in the master branch, and PyPI releases could reference the issues fixed in each version. People with the same issue could still find this ticket and understand the release by comparing the Git tags in logs. But @1st1 may have a different opinion
1st1 commentedon Feb 5, 2021
Closing as soon as it's fixes in master is IMO ok, but then I typically add a comment saying when the release is scheduled and then go back and update the issue again once the release has been made.
lovasoa commentedon Feb 6, 2021
Yes, I also think this is the right thing to do. So, is there a new release planned ? The last one was in November 2019.
fantix commentedon Feb 7, 2021
Yes! I'll push for a new release after the first 3 columns on this board are cleared.
fantix commentedon Feb 10, 2021
The fix is included in 0.15.0.
lovasoa commentedon Feb 10, 2021
Thank you @fantix !
Add a regression test for large stderr reading