Skip to content

Can't build boost with python 3 on ubuntu 18.04 #33

New issue
New issue
Closed
Closed
Can't build boost with python 3 on ubuntu 18.04#33

Description

@hstarmans
hstarmans
opened on Oct 11, 2019

I am able to build and run boost with python 2 on ubuntu 18.04. I am not able to build and run boost with python 3.
My steps are as follows;

  1. Download boost
    I tried both boost 1.71 and 1.67 both in the end fail. Example here is given for 1.71.
cd /home/user/
wget https://dl.bintray.com/boostorg/release/1.71.0/source/boost_1_71_0.tar.bz2
tar --bzip2 -xf boost_1_71_0.tar.bz2
  1. Build boost against python 3
    The prefix folder is where the python library gets written
cd /home/user/boost_1_71_0
./bootstrap.sh -with-libraries=python -with-python-version=3.6 --prefix=/home/user/temp/
./b2 &./b2 install
  1. Download this library
cd /home/user/
git clone https://github.com/TNG/boost-python-examples
  1. Create a virtual environment so python points to python 3
cd boost-python-examples
python3 -m venv environment
source /environment/bin/activate
  1. Run cmake with flags
    I read this stack over flow post
    As a result I run the following commands;
set -x
rm -rf build
mkdir build
cd build
cmake -DPYTHON_VERSION_SUFFIX=-py3 -DCMAKE_BUILD_TYPE=DEBUG -DBOOST_ROOT=/home/user/temp/ -DBoost_NO_SYSTEM_PATHS=TRUE ..

Result -- Could NOT find Boost
Also tried -DBOOST_ROOT=/home/user/temp/boost_1_71_0 and several other options to no avail.

I have also tried to modify the cmakecache.txt from the python 2 configuration into variables suited for python 3.
This eventually leads to the error;

relocation R_X86_64_PC32 against symbol `PyLong_Type' can not be used when making a shared object; recompile with -fPIC

I also obtain this error when i tried to run examples using bjam.

Activity

  1. hstarmans
    changed the title [-]Can't build boost with python 3 on ubuntu[/-] [+]Can't build boost with python 3 on ubuntu 18.04[/+] on Oct 11, 2019

  2. Achimh3011 commented on Oct 11, 2019

    @Achimh3011
    Achimh3011
    on Oct 11, 2019
    Member

    Did you check which python version suffix was used in the Boost compilation (if any)? They are not standardized and might even be omitted in manual compilation of the library.

  3. hstarmans commented on Oct 11, 2019

    @hstarmans
    hstarmans
    on Oct 11, 2019
    Author

    Thanks for the quick reply. I made it work in python 3 by modifying the cmakecache.
    I have also fixed it by modifying the CMakeList.txt. I believe there is a bug; see next post.

    First i remove all instances of boost and made sure cmake could not find any instance.

    dpkg -S /usr/include/boost/version.hpp

    I removed all these packages. I installed the Ubuntu default boost packages.

    sudo apt install libboost-dev libboost-all-dev

    Create a directory, run cmake and then make. It detects python 2 and boost. I checked I was able to run examples for this python 2 version.
    I then made a new directory ran cmake again

    mkdir build2 && cd build 2 && cmake ..

    I hacked the cache of cmake CMakeCache.txt in the build folder and than ran make; it worked.

    # There is data before this but not relevant so I didn't post it on Github

########################
# EXTERNAL cache entries
########################

//The directory containing a CMake configuration file for Boost.
Boost_DIR:PATH=Boost_DIR-NOTFOUND

//Path to a file.
Boost_INCLUDE_DIR:PATH=/usr/include

//Boost library directory DEBUG
Boost_LIBRARY_DIR_DEBUG:PATH=/usr/lib/x86_64-linux-gnu

//Boost library directory RELEASE
Boost_LIBRARY_DIR_RELEASE:PATH=/usr/lib/x86_64-linux-gnu

//Boost python library (debug)
Boost_PYTHON_LIBRARY_DEBUG:FILEPATH=/usr/lib/x86_64-linux-gnu/libboost_python3.so

//Boost python library (release)
Boost_PYTHON_LIBRARY_RELEASE:FILEPATH=/usr/lib/x86_64-linux-gnu/libboost_python3.so

//Path to a program.
CMAKE_AR:FILEPATH=/usr/bin/ar

//Choose the type of build, options are: None(CMAKE_CXX_FLAGS or
// CMAKE_C_FLAGS used) Debug Release RelWithDebInfo MinSizeRel.
CMAKE_BUILD_TYPE:STRING=

//Enable/Disable color output during build.
CMAKE_COLOR_MAKEFILE:BOOL=ON

//CXX compiler
CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++

//A wrapper around 'ar' adding the appropriate '--plugin' option
// for the GCC compiler
CMAKE_CXX_COMPILER_AR:FILEPATH=/usr/bin/gcc-ar-7

//A wrapper around 'ranlib' adding the appropriate '--plugin' option
// for the GCC compiler
CMAKE_CXX_COMPILER_RANLIB:FILEPATH=/usr/bin/gcc-ranlib-7

//Flags used by the compiler during all build types.
CMAKE_CXX_FLAGS:STRING=

//Flags used by the compiler during debug builds.
CMAKE_CXX_FLAGS_DEBUG:STRING=-g

//Flags used by the compiler during release builds for minimum
// size.
CMAKE_CXX_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG

//Flags used by the compiler during release builds.
CMAKE_CXX_FLAGS_RELEASE:STRING=-O3 -DNDEBUG

//Flags used by the compiler during release builds with debug info.
CMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG

//C compiler
CMAKE_C_COMPILER:FILEPATH=/usr/bin/cc

//A wrapper around 'ar' adding the appropriate '--plugin' option
// for the GCC compiler
CMAKE_C_COMPILER_AR:FILEPATH=/usr/bin/gcc-ar-7

//A wrapper around 'ranlib' adding the appropriate '--plugin' option
// for the GCC compiler
CMAKE_C_COMPILER_RANLIB:FILEPATH=/usr/bin/gcc-ranlib-7

//Flags used by the compiler during all build types.
CMAKE_C_FLAGS:STRING=

//Flags used by the compiler during debug builds.
CMAKE_C_FLAGS_DEBUG:STRING=-g

//Flags used by the compiler during release builds for minimum
// size.
CMAKE_C_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG

//Flags used by the compiler during release builds.
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG

//Flags used by the compiler during release builds with debug info.
CMAKE_C_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG

//Flags used by the linker.
CMAKE_EXE_LINKER_FLAGS:STRING=

//Flags used by the linker during debug builds.
CMAKE_EXE_LINKER_FLAGS_DEBUG:STRING=

//Flags used by the linker during release minsize builds.
CMAKE_EXE_LINKER_FLAGS_MINSIZEREL:STRING=

//Flags used by the linker during release builds.
CMAKE_EXE_LINKER_FLAGS_RELEASE:STRING=

//Flags used by the linker during Release with Debug Info builds.
CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO:STRING=

//Enable/Disable output of compile commands during generation.
CMAKE_EXPORT_COMPILE_COMMANDS:BOOL=OFF

//Install path prefix, prepended onto install directories.
CMAKE_INSTALL_PREFIX:PATH=/usr/local

//Path to a program.
CMAKE_LINKER:FILEPATH=/usr/bin/ld

//Path to a program.
CMAKE_MAKE_PROGRAM:FILEPATH=/usr/bin/make

//Flags used by the linker during the creation of modules.
CMAKE_MODULE_LINKER_FLAGS:STRING=

//Flags used by the linker during debug builds.
CMAKE_MODULE_LINKER_FLAGS_DEBUG:STRING=

//Flags used by the linker during release minsize builds.
CMAKE_MODULE_LINKER_FLAGS_MINSIZEREL:STRING=

//Flags used by the linker during release builds.
CMAKE_MODULE_LINKER_FLAGS_RELEASE:STRING=

//Flags used by the linker during Release with Debug Info builds.
CMAKE_MODULE_LINKER_FLAGS_RELWITHDEBINFO:STRING=

//Path to a program.
CMAKE_NM:FILEPATH=/usr/bin/nm

//Path to a program.
CMAKE_OBJCOPY:FILEPATH=/usr/bin/objcopy

//Path to a program.
CMAKE_OBJDUMP:FILEPATH=/usr/bin/objdump

//Value Computed by CMake
CMAKE_PROJECT_NAME:STATIC=Project

//Path to a program.
CMAKE_RANLIB:FILEPATH=/usr/bin/ranlib

//Flags used by the linker during the creation of dll's.
CMAKE_SHARED_LINKER_FLAGS:STRING=

//Flags used by the linker during debug builds.
CMAKE_SHARED_LINKER_FLAGS_DEBUG:STRING=

//Flags used by the linker during release minsize builds.
CMAKE_SHARED_LINKER_FLAGS_MINSIZEREL:STRING=

//Flags used by the linker during release builds.
CMAKE_SHARED_LINKER_FLAGS_RELEASE:STRING=

//Flags used by the linker during Release with Debug Info builds.
CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO:STRING=

//If set, runtime paths are not added when installing shared libraries,
// but are added when building.
CMAKE_SKIP_INSTALL_RPATH:BOOL=NO

//If set, runtime paths are not added when using shared libraries.
CMAKE_SKIP_RPATH:BOOL=NO

//Flags used by the linker during the creation of static libraries.
CMAKE_STATIC_LINKER_FLAGS:STRING=

//Flags used by the linker during debug builds.
CMAKE_STATIC_LINKER_FLAGS_DEBUG:STRING=

//Flags used by the linker during release minsize builds.
CMAKE_STATIC_LINKER_FLAGS_MINSIZEREL:STRING=

//Flags used by the linker during release builds.
CMAKE_STATIC_LINKER_FLAGS_RELEASE:STRING=

//Flags used by the linker during Release with Debug Info builds.
CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO:STRING=

//Path to a program.
CMAKE_STRIP:FILEPATH=/usr/bin/strip

//If this value is on, makefiles will be generated without the
// .SILENT directive, and all commands will be echoed to the console
// during the make.  This is useful for debugging only. With Visual
// Studio IDE projects all commands are done without /nologo.
CMAKE_VERBOSE_MAKEFILE:BOOL=FALSE

//Add module classes
PYTHON_ENABLE_MODULE_classes:BOOL=ON

//Add module ctor
PYTHON_ENABLE_MODULE_ctor:BOOL=ON

//Add module hello
PYTHON_ENABLE_MODULE_hello:BOOL=ON

//Add module inheritance
PYTHON_ENABLE_MODULE_inheritance:BOOL=ON

//Add module iterators
PYTHON_ENABLE_MODULE_iterators:BOOL=ON

//Add module member
PYTHON_ENABLE_MODULE_member:BOOL=ON

//Add module myexceptions
PYTHON_ENABLE_MODULE_myexceptions:BOOL=ON

//Add module mymodule
PYTHON_ENABLE_MODULE_mymodule:BOOL=ON

//Add module operators
PYTHON_ENABLE_MODULE_operators:BOOL=ON

//Add module overload
PYTHON_ENABLE_MODULE_overload:BOOL=ON

//Add module policies
PYTHON_ENABLE_MODULE_policies:BOOL=ON

//Add module virtual
PYTHON_ENABLE_MODULE_virtual:BOOL=ON

//Path to a program.
PYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3

//Path to a file.
PYTHON_INCLUDE_DIR:PATH=/usr/include/python3.6

//Path to a library.
PYTHON_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libpython3.6m.so

//Path to a library.
PYTHON_LIBRARY_DEBUG:FILEPATH=PYTHON_LIBRARY_DEBUG-NOTFOUND

//Add module classes shared
PYTHON_MODULE_classes_BUILD_SHARED:BOOL=ON

//Add module ctor shared
PYTHON_MODULE_ctor_BUILD_SHARED:BOOL=ON

//Add module hello shared
PYTHON_MODULE_hello_BUILD_SHARED:BOOL=ON

//Add module inheritance shared
PYTHON_MODULE_inheritance_BUILD_SHARED:BOOL=ON

//Add module iterators shared
PYTHON_MODULE_iterators_BUILD_SHARED:BOOL=ON

//Add module member shared
PYTHON_MODULE_member_BUILD_SHARED:BOOL=ON

//Add module myexceptions shared
PYTHON_MODULE_myexceptions_BUILD_SHARED:BOOL=ON

//Add module mymodule shared
PYTHON_MODULE_mymodule_BUILD_SHARED:BOOL=ON

//Add module operators shared
PYTHON_MODULE_operators_BUILD_SHARED:BOOL=ON

//Add module overload shared
PYTHON_MODULE_overload_BUILD_SHARED:BOOL=ON

//Add module policies shared
PYTHON_MODULE_policies_BUILD_SHARED:BOOL=ON

//Add module virtual shared
PYTHON_MODULE_virtual_BUILD_SHARED:BOOL=ON

//Value Computed by CMake
Project_BINARY_DIR:STATIC=/home/starmans/boost-python-examples/build2

//Value Computed by CMake
Project_SOURCE_DIR:STATIC=/home/starmans/boost-python-examples

//Dependencies for the target
hello_LIB_DEPENDS:STATIC=general;/usr/lib/x86_64-linux-gnu/libboost_python3.so;general;/usr/lib/x86_64-linux-gnu/libpython3.6m.so;

//I have remove the other targets and everything which comes after this.. To minimize output on //Github

    So the remaining questions is how do you get cmake into coming up with this output !?

  4. hstarmans commented on Oct 11, 2019

    @hstarmans
    hstarmans
    on Oct 11, 2019
    Author

    Okay let's conclude this. Let's have a look at the provided CMakeList.txt.
    In line 3 it states;

    FIND_PACKAGE(PythonInterp)

    The problem with this is that it will default to Python 2 and never Python 3. Creating a virtual environment to circumvent this is a bit hacky.
    I modified this to

    FIND_PACKAGE(PythonInterp 3)

    Then as I am on unix and not apple I arrive at line 8. Here we have the following;

    FIND_PACKAGE(Boost COMPONENTS python${PYTHON_VERSION_SUFFIX})

    There is an obvious typo here, it should be;

    FIND_PACKAGE(Boost COMPONENTS python${PYTHON_VERSION_MAJOR}

    The installation works with these two modifications.

    The question that remains is can't you pass a flag to cmake that it looks for python 2 or 3?
    Personally, I think it should default to Python 3. If it can not find python3 it should look for python 2.

  5. Achimh3011 commented on Oct 13, 2019

    @Achimh3011
    Achimh3011
    on Oct 13, 2019
    Member

    So for Ubuntu 18.04 the version suffix is simply a 3? Then that needs to be considered in build.sh. If you want, you could add a switch for enforcing python3 there as well.

  6. hstarmans commented on Oct 13, 2019

    @hstarmans
    hstarmans
    on Oct 13, 2019
    Author

    A suffix is a letter or a group of letters attached to the end of a word to form a new word. So Python 3.7 here 3 is version major and 7 is the suffix. This mistake isn't in the other cases where you have;

    FIND_PACKAGE(Boost COMPONENTS python${PYTHON_VERSION_MAJOR}${PYTHON_VERSION_MINOR})

    In the build.sh I would remove -DPYTHON_VERSION_SUFFIX

    cmake  -DCMAKE_BUILD_TYPE=DEBUG .. && make && make test

    I would start the cmake file with the code shown below, as it defaults to 3.

    FIND_PACKAGE(PythonInterp 3)
     if (NOT PYTHONINTERP_FOUND)
          FIND_PACKAGE(PythonInterp)

  7. hstarmans commented on Oct 14, 2019

    @hstarmans
    hstarmans
    on Oct 14, 2019
    Author

    The issue is closed from my side, I have decided to move to pybind as it is more lightweight than boost. I could also find a cmake, setup.py example. This works for now.

  8. hstarmans
    closed this as completedon Oct 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions