OpenBLAS on Ryzen 2500u

[Ry7en]

Following the instructions on the "Building a 64-bit version of the Embedded Learning Library (ELL) on Windows" page I reached an impasse at the "Invoke CMake as follows" instruction ("-- Unknown processor, disabling OpenBLAS"). A google search led me to believe that this is caused by my laptops processor, an AMD 2500u, not being included in the CMake\OpenBLASSetup.cmake script, but I couldn't find the instructions, suggested on a thread on the issues page here, required to build OpenBLAS on my computer. Any help would be appreciated.

[lovettchris]

clone https://github.com/xianyi/OpenBLAS, then:

mkdir build
cd build
cmake -G "Visual Studio 15 2017 Win64" ..
cmake --build . --config Release

This will give you an OpenBlas build that works for your computer. The output of this build is in OpenBLAS\build\lib\release where you will see libopenblas.dll. Then run this from an Admin command prompt:

cmake --build . --config Release --target INSTALL

This will put the installed bits in "C:/Program Files/OpenBLAS" and the next step is helping the ELL build find this build of OpenBlas. The easiest way might be something like this before the FindPackage call:

set(BLAS_DLL_DIR "C:/Program Files/OpenBLAS/bin")
set(BLAS_INCLUDE_SEARCH_PATHS ${BLAS_PACKAGE_DIR}/include/)
set(BLAS_LIB_SEARCH_PATHS ${BLAS_PACKAGE_DIR}/lib/)

[lovettchris]

It is a bit more work to get OpenBlasSetup.cmake to find this version, see attached updates.

OpenBLASSetup.zip

[Ry7en]

Thank you very much for your prompt reply; after running "cmake -G "Visual Studio 15 2017 Win64" .." the output contained a number of warnings, namely:
a "CMake Deprecation Warning";
multiple "Policy CMP0054 is not set" warnings;
a "Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)" warning.

(py36) C:\Users\HP\OpenBLAS\build>cmake -G "Visual Studio 15 2017 Win64" ..
-- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.17134.
-- The C compiler identification is MSVC 19.16.27027.1
-- The ASM compiler identification is MSVC
-- Found assembler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
CMake Warning at CMakeLists.txt:59 (message):
  CMake support is experimental.  It does not yet support all build options
  and may not produce the same Makefiles that OpenBLAS ships with.


-- GEMM multithread threshold set to 4.
-- Multi-threading enabled with 8 threads.
CMake Deprecation Warning at C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.12/Modules/CMakeForceCompiler.cmake:97 (message):
  The CMAKE_FORCE_Fortran_COMPILER macro is deprecated.  Instead just set
  CMAKE_Fortran_COMPILER and allow CMake to identify the compiler.
Call Stack (most recent call first):
  cmake/f_check.cmake:27 (CMAKE_FORCE_Fortran_COMPILER)
  cmake/prebuild.cmake:84 (include)
  cmake/system.cmake:141 (include)
  CMakeLists.txt:62 (include)


CMake Warning (dev) at cmake/prebuild.cmake:298 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "MSVC" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.
Call Stack (most recent call first):
  cmake/system.cmake:141 (include)
  CMakeLists.txt:62 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

MSVC
-- Running getarch
-- GETARCH results:
CORE=generic
LIBCORE=generic
NUM_CORES=8
GENERIC=1
L1_DATA_SIZE=32768
L1_DATA_LINESIZE=128
L2_SIZE=512488
L2_LINESIZE=128
DTB_DEFAULT_ENTRIES=128
DTB_SIZE=4096
L2_ASSOCIATIVE=8
MAKE += -j 8

-- Building Single Precision
-- Building Double Precision
-- Building Complex Precision
-- Building Double Complex Precision
-- Reading vars from C:/Users/HP/OpenBLAS/kernel/x86_64/KERNEL...
-- Reading vars from C:/Users/HP/OpenBLAS/kernel/x86_64/KERNEL.generic...
CMake Warning (dev) at kernel/CMakeLists.txt:124 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "GENERIC" will no longer be dereferenced when the
  policy is set to NEW.  Since the policy is not set the OLD behavior will be
  used.
Call Stack (most recent call first):
  kernel/CMakeLists.txt:546 (build_core)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
CMake Warning (dev) at utest/CMakeLists.txt:4 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "MSVC" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating openblas_config.h in include/openblas
-- Generating f77blas.h in include/openblas
-- Generating cblas.h in include/openblas
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/HP/OpenBLAS/build

In my previous attempt to resolve the problem of getting OpenBLAS to work I tried following the instructions for a "Native (MSVC) ABI" build (https://github.com/xianyi/OpenBLAS/wiki/How-to-use-OpenBLAS-in-Microsoft-Visual-Studio), and I recall that some packages installed previously during the ELL installation were up and down-graded. Perhaps this is causing the PkgConfig warning?

[lovettchris]

You can safely ignore all those warnings, go ahead with the build step:

cmake --build . --config Release

[Ry7en]

The build step has completed with no errors, but 515 warnings, which mostly seem to be code C4244:

conversion from 'BLASLONG' to 'int', possible loss of data

[lovettchris]

Great, now for the install step from an admin command prompt:

cmake --build . --config Release --target INSTALL

[Ry7en]

The install proceeded with no errors, but for the next step should I skip the last instruction in your first post, and replace the OpenBLASSetup.cmake in my ELL/CMake folder with the one you attached in your second post?

[lovettchris]

yes, replace OpenBLASSetup.cmake in your ELL/Cmake with the one I attached earlier, then delete any ELL/build folder you may have created earlier, then follow the ELL build instrucrtions.

[Ry7en]

After following the "Building ELL" section of the installation guide and running "cmake -G "Visual Studio 15 2017 Win64" ..", the following was output, including the "-- Unknown processor, disabling OpenBLAS" warning :

(py36) C:\Users\HP\ELL\build>cmake -G "Visual Studio 15 2017 Win64" ..
-- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.17134.
-- The C compiler identification is MSVC 19.16.27027.1
-- The CXX compiler identification is MSVC 19.16.27027.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- ELL version 2.4.0
-- Configuring tests to use TEST_MODELS_REPO at: https://github.com/Microsoft/ell-test-models
-- PYTHONINTERP_FOUND=FALSE
-- PYTHON_EXECUTABLE=C:/Users/HP/Miniconda3/envs/py36/python.exe
-- PYTHON_VERSION_STRING=3.6.8
-- PYTHON_VERSION_MAJOR=3
-- PYTHONLIBS_FOUND=FALSE
-- PYTHON_LIBRARIES=C:/Users/HP/Miniconda3/envs/py36/libs/python36.lib
-- PYTHON_INCLUDE_PATH=C:/Users/HP/Miniconda3/envs/py36/include
-- PYTHON_INCLUDE_DIRS=C:/Users/HP/Miniconda3/envs/py36/include
-- PYTHON_DEBUG_LIBRARIES=PYTHON_DEBUG_LIBRARY-NOTFOUND
-- PYTHONLIBS_VERSION_STRING=3.6.8
-- PYTHONINTERP_FOUND=TRUE
-- PYTHON_EXECUTABLE=C:/Users/HP/Miniconda3/envs/py36/python.exe
-- PYTHON_VERSION_STRING=3.6.8
-- PYTHON_VERSION_MAJOR=3
-- PYTHONLIBS_FOUND=TRUE
-- PYTHON_LIBRARIES=C:/Users/HP/Miniconda3/envs/py36/libs/python36.lib
-- PYTHON_INCLUDE_PATH=C:/Users/HP/Miniconda3/envs/py36/include
-- PYTHON_INCLUDE_DIRS=C:/Users/HP/Miniconda3/envs/py36/include
-- PYTHON_DEBUG_LIBRARIES=PYTHON_DEBUG_LIBRARY-NOTFOUND
-- PYTHONLIBS_VERSION_STRING=3.6.8
CMake Warning at CMake/Flake8.cmake:7 (message):
  ### could not find 'flake8', please run 'pip install flake8'
Call Stack (most recent call first):
  CMakeLists.txt:86 (include)


-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Processor family: 23, model: 17
CMake Error at CMake/OpenBLASSetup.cmake:171 (get_processor_mapping):
  get_processor_mapping Macro invoked with incorrect arguments for macro
  named: get_processor_mapping
Call Stack (most recent call first):
  CMakeLists.txt:145 (include)


-- Unknown processor, disabling OpenBLAS
-- BLAS library not found
-- Found LLVM 6.0.1
-- Using LLVMConfig.cmake in: C:/Users/HP/ELL/external/LLVMNativeWindowsLibs.x64.6.0.1/llvm-6.0/lib/cmake/llvm
-- Found SWIG: C:/Users/HP/ELL/external/swigwintools.3.0.12/tools/swigwin-3.0.12/swig.exe (found suitable version "3.0.12", minimum required is "3.0.12")
-- Found SWIG_EXECUTABLE=C:/Users/HP/ELL/external/swigwintools.3.0.12/tools/swigwin-3.0.12/swig.exe
-- Skipping pitest because one of RPI_CLUSTER, RPI_PASSWORD, RPI_KEY, TEST_MODELS_REPO is missing
-- Creating wrappers for python
CMake Deprecation Warning at C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.12/Modules/UseSWIG.cmake:492 (message):
  SWIG_ADD_MODULE is deprecated.  Use SWIG_ADD_LIBRARY instead.
Call Stack (most recent call first):
  CMake/CommonInterfaces.cmake:129 (swig_add_module)
  CMake/CommonInterfaces.cmake:160 (generate_interface_module)
  interfaces/python/CMakeLists.txt:21 (generate_interface)


-- Creating wrappers for javascript
-- Creating wrappers for xml
-- Writing: C:/Users/HP/ELL/build/config.json
-- Configuring incomplete, errors occurred!
See also "C:/Users/HP/ELL/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/HP/ELL/build/CMakeFiles/CMakeError.log".

[lovettchris]

CMake Error at CMake/OpenBLASSetup.cmake:171 (get_processor_mapping):
  get_processor_mapping Macro invoked with incorrect arguments for macro
  named: get_processor_mapping

Huh, what version of cmake are you using? This is the one I'm using:

C:\Program Files\CMake\bin\cmake.exe
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe

(ell) c:\git\ELL\ELL>cmake --version
cmake version 3.10.0

[Ry7en]

I get:

C:\Users\HP\ELL>cmake --version
cmake version 3.12.18081601-MSVC_2

CMake suite maintained and supported by Kitware (kitware.com/cmake).

I believe that CMake may have been updated during my aforementioned attempt to follow the "Native (MSVC) ABI" OpenBLAS build tutorial.

[lovettchris]

Hmm, I just tried my modified OpenBLASSetup.cmake on another machine with the exact same version of cmake (cmake version 3.12.18081601-MSVC_2) and it works fine... can you run the following on your machine and report the output?

(ell) d:\temp\ELL\ELL\CMake>set proc
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 79 Stepping 1, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=4f01

I suspect we have a bug in our get_processor_mapping macro... oh, and I need the output of this one too:

(ell) d:\temp\ELL\ELL\CMake>reg query HKEY_LOCAL_MACHINE\Hardware\Description\System\CentralProcessor\0

HKEY_LOCAL_MACHINE\Hardware\Description\System\CentralProcessor\0
    Component Information    REG_BINARY    00000000000000000000000000000000
    Identifier    REG_SZ    Intel64 Family 6 Model 79 Stepping 1
    Configuration Data    REG_FULL_RESOURCE_DESCRIPTOR    FFFFFFFFFFFFFFFF0000000000000000
    ProcessorNameString    REG_SZ    Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
    VendorIdentifier    REG_SZ    GenuineIntel
    FeatureSet    REG_DWORD    0x3d1b3fff
    ~MHz    REG_DWORD    0x82f
    Update Revision    REG_BINARY    000000002E00000B
    Update Status    REG_DWORD    0x7
    Previous Update Revision    REG_BINARY    000000002E00000B
    Platform Specific Field 1    REG_DWORD    0x1

[Ry7en]

In the first example I get:

C:\Users\HP\ELL\CMake>set proc
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=AMD64 Family 23 Model 17 Stepping 0, AuthenticAMD
PROCESSOR_LEVEL=23
PROCESSOR_REVISION=1100

Whilst the second has:

C:\Users\HP\ELL\CMake>reg query HKEY_LOCAL_MACHINE\Hardware\Description\System\CentralProcessor\0

HKEY_LOCAL_MACHINE\Hardware\Description\System\CentralProcessor\0
    Component Information    REG_BINARY    00000000000000000000000000000000
    Identifier    REG_SZ    AMD64 Family 23 Model 17 Stepping 0
    Configuration Data    REG_FULL_RESOURCE_DESCRIPTOR    FFFFFFFFFFFFFFFF0000000000000000
    ProcessorNameString    REG_SZ    AMD Ryzen 5 PRO 2500U w/ Radeon Vega Mobile Gfx
    VendorIdentifier    REG_SZ    AuthenticAMD
    FeatureSet    REG_DWORD    0x3c3b3dff
    ~MHz    REG_DWORD    0x7cc
    Update Revision    REG_BINARY    0B10100800000000
    Update Status    REG_DWORD    0x1
    Previous Update Revision    REG_BINARY    0B10100800000000
    Platform Specific Field1    REG_DWORD    0x810100b

[lovettchris]

Oh, hang on, the OpenBLASSetup.cmake shouldn't be getting to line 171. It should be finding your OpenBlas installation in c:\program files\openblas… but this output means it did not find that:

-- BLAS library not found

So we need to debug why the new lines I added 108-121 didn't succeed in finding your openblas install... running this line and attaching a zipped version of the trace.log file will help with that:

cmake -G "Visual Studio 15 2017 Win64" .. --trace > trace.log 2>&1

[Ry7en]

I hope this is what was required:

trace.zip