test_numpy_interoperability.test_np_array_function_protocol broken with numpy 1.18

[leezu]

Description

test_numpy_interoperability.test_np_array_function_protocol is broken with numpy 1.18 ?

Error Message

======================================================================
ERROR: test_numpy_interoperability.test_np_array_function_protocol
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python37\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\jenkins_slave\workspace\ut-python-gpu\tests\python\unittest\common.py", line 221, in test_new
    orig_test(*args, **kwargs)
  File "C:\jenkins_slave\workspace\ut-python-gpu\windows_package\python\mxnet\util.py", line 321, in _with_np_shape
    return func(*args, **kwargs)
  File "C:\jenkins_slave\workspace\ut-python-gpu\windows_package\python\mxnet\util.py", line 505, in _with_np_array
    return func(*args, **kwargs)
  File "C:\jenkins_slave\workspace\ut-python-gpu\windows_package\python\mxnet\numpy_dispatch_protocol.py", line 60, in _run_with_array_func_proto
    .format(func.__name__, str(e)))
RuntimeError: Running function test_np_array_function_protocol with NumPy array function protocol failed with exception Non-string object detected for the array ordering. Please pass in 'C', 
'F', 'A', or 'K' instead
-------------------- >> begin captured stdout << ---------------------

To Reproduce

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/master/1414/pipeline
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-cpu/detail/master/1418/pipeline
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/master/1433/pipeline
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/master/1427/pipeline

@reminisce @haojin2

[haojin2]

@leezu I'm seeing:

======================================================================

ERROR: Failure: ImportError (No module named 'numpy.testing.decorators')

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/nose/failure.py", line 39, in runTest

    raise self.exc_val.with_traceback(self.tb)

  File "/usr/local/lib/python3.5/dist-packages/nose/loader.py", line 418, in loadTestsFromName

    addr.filename, addr.module)

  File "/usr/local/lib/python3.5/dist-packages/nose/importer.py", line 47, in importFromPath

    return self.importFromDir(dir_path, fqname)

  File "/usr/local/lib/python3.5/dist-packages/nose/importer.py", line 94, in importFromDir

    mod = load_module(part_fqname, fh, filename, desc)

  File "/usr/lib/python3.5/imp.py", line 234, in load_module

    return load_source(name, filename, file)

  File "/usr/lib/python3.5/imp.py", line 172, in load_source

    module = _load(spec)

  File "<frozen importlib._bootstrap>", line 693, in _load

  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked

  File "<frozen importlib._bootstrap_external>", line 665, in exec_module

  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed

  File "/work/mxnet/tests/python/gpu/test_operator_gpu.py", line 36, in <module>

    from test_numpy_ndarray import *

  File "/work/mxnet/tests/python/gpu/../unittest/test_numpy_ndarray.py", line 33, in <module>

    import scipy.stats as ss

  File "/usr/local/lib/python3.5/dist-packages/scipy/stats/__init__.py", line 345, in <module>

    from .morestats import *

  File "/usr/local/lib/python3.5/dist-packages/scipy/stats/morestats.py", line 12, in <module>

    from numpy.testing.decorators import setastest

ImportError: No module named 'numpy.testing.decorators'
----------------------------------------------------------------------

at http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/master/1433/pipeline/.

[haojin2]

@leezu Please remove unrelated links so that we don't lose focus on the issue, and open separate issues for those failure caused by other parts of the system. I'm working on reproducing the interoperability failure now.

[haojin2]

@leezu Fix for the interoperability issue delivered in #17155, please do open separate issues for other problems.

[leezu]

@haojin2, sorry to link multiple broken CI steps. I didn't investigate the exact set of issues further after linking the failure to the numpy 1.18 release. I will be more careful in the future.

[samskalicky]

@apeforest assign [@reminisce ]

[reminisce]

With the latest master commit and numpy 1.18, I have no problem to run test_numpy_interoperability.py on Ubuntu.

[leezu]

@reminisce there may be further numpy 1.18 incompatibilities. If all are resolved, 318d9c7#diff-69f4d73bfad59a614d2c730c4f327db6R29 should be reverted to verify in the CI environment

Direct link: https://github.com/apache/incubator-mxnet/blob/44784c2554f0330b94d8bde411394da89405c397/ci/docker/install/requirements#L29

[reminisce]

@leezu I submitted the reverting PR here: #17200. It has passed CI. Could you help to review and merge if possible? Thanks.

[leezu]

Thank you