Repository navigation
building wheel from sdist fails #320
Description
- uvloop version: 0.14.0
- Python version: 3.6 and 3.7
- Platform: Linux (CentOS 7
- Can you reproduce the bug with
PYTHONASYNCIODEBUGin env?: N/A - Does uvloop behave differently from vanilla asyncio? How?: N/A
What I did:
pip wheel --global-option build_ext --global-option --inplace uvloop==0.14.0
What I expected:
The command would build uvloop and bundle it as a wheel for me.
What I saw:
The command fails with error: don't know how to compile C/C++ code on platform 'posix' with '<distutils.unixccompiler.UnixCCompiler object at 0x7f054bc1fa10>' compiler after build_ext runs (e.g. during the processing of bdist_wheel).
The (heavily) snipped output from the above command is:
</path/to/venv>/lib/python3.7/site-packages/pip/_internal/commands/wheel.py:117: UserWarning: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
cmdoptions.check_install_build_global(options)
Looking in indexes: https://pypi.org/simple, https://<private repo>/simple
Collecting uvloop==0.14.0
Using cached uvloop-0.14.0.tar.gz (2.0 MB)
Building wheels for collected packages: uvloop
Building wheel for uvloop (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: </path/to/venv>/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-12kxwkdh/uvloop/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-12kxwkdh/uvloop/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' build_ext --inplace bdist_wheel -d /tmp/pip-wheel-exvxqjuv
cwd: /tmp/pip-wheel-12kxwkdh/uvloop/
Complete output (226 lines):
running build_ext
... <lots of successful build output
building 'uvloop.loop' extension
... <more successful built output>
copying build/lib.linux-x86_64-3.7/uvloop/loop.cpython-37m-x86_64-linux-gnu.so -> uvloop
running bdist_wheel
running build
running build_py
... <lots of copying/writing for building the wheel>
copying uvloop/includes/uv.pxd -> build/lib.linux-x86_64-3.7/uvloop/includes
running build_ext
error: don't know how to compile C/C++ code on platform 'posix' with '<distutils.unixccompiler.UnixCCompiler object at 0x7f054bc1fa10>' compiler
----------------------------------------
ERROR: Failed building wheel for uvloop
Running setup.py clean for uvloop
Failed to build uvloop
ERROR: Failed to build one or more wheels
Other observations:
The pip wheel command essentially downloads your sdist, unwinds it, cd's to that directory, and runs python setup.py build_ext --inplace bdist_wheel This mimics your make file other than the fact that build_ext and bdist_wheel are on one command line.
If I manually run python setup.py build_ext --inplace && python setup.py bdist_wheel, then the wheel is successfully built. I can't tell exactly how, but it seems like something is building the extension in bdist_wheel instead of simply inheriting what was build by build_ext. That, in turn, is triggering what may be a latent distutils issue that I documented here.
Activity
elprans commentedon Mar 17, 2020
Do you have python3-devel installed?
televi commentedon Mar 18, 2020
Yes I do. I'm able to reproduce this at will on both CentOS 7.7 and Mac OS X 10.15.3 (Catalina) with Python 3.6, 3.7, and 3.8.
Here's the full set of commands and output that I used on CentOS 7.7 with Python 3.6. Note: I'm able to compile dozens of other Python packages using the same command both with and without Cython.
In the output below, you'll see that it compiles fine, then tries to run bdist_wheel (search for "running bdist_wheel") and then tries to compile again and that fails. Note the error at the end. That should not point at a UnixCCompiler object - it should be a string. That string gets used as a key to a dict and, in the case of running on CentOS, should be "unix". If it were, this would work.
What seems to be happening is that (somehow) your setup.py file is passing a compiler object back to a method to create a new compiler object. I've not seen any other Cython package we build using this command do that so I don't know why that is happening.
televi commentedon Mar 18, 2020
I think I may have found it.
In your setup.py, you're short-circuiting the call to super().initialize_options() here. That means you're forcing subsequent commands to inherit variables that they expect to set for themselves (like self.compiler, which is what is causing the build failure). The same applies to finalize_options().
If you simply want to avoid your options from being reset, you could move the 4 lines that set your options into the body of that if and always call super()initialize_options().
I'm not sure why you're avoiding those calls to super().foo() in initialize_options() and finalize_options(), but those seem to be what is messing up builds with multiple commands on the same line.
elprans commentedon Mar 19, 2020
Hm. Good catch. I'll take a look and see if this hack is still necessary.
televi commentedon Aug 19, 2022
I know this is old, but it still fails. Any luck deciding on whether that hack was still required? As it stands, uvloop still can't be built from sdist 😦
fantix commentedon Sep 9, 2022
I think we can drop this hack now, as
test_suiteis already dropped insetup.py, refs #84Partially revert "Fix running of setup.py test with recent setuptools. (
Partially revert "Fix running of setup.py test with recent setuptools. (