Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

gluon.data.DataLoader when set num_workers>0, Exceptions always occurred #21084

Description

@CookingANut

Description

train_iter = gluon.data.DataLoader(
mnist_train.transform_first(transformer),
batch_size,
shuffle=True,
num_workers=4
)

when i set num_workers = 4
seems code can run through, but will appear below exception:

Exception ignored in: <function DataLoader.del at 0x7f1d7fe165e0>
Traceback (most recent call last):
File "/home/daemonh/miniconda3/envs/d2l/lib/python3.8/site-packages/mxnet/gluon/data/dataloader.py", line 720, in del
AttributeError: 'NoneType' object has no attribute 'Pool'
Exception ignored in: <function Pool.del at 0x7f1d7fe84670>
Traceback (most recent call last):
File "/home/daemonh/miniconda3/envs/d2l/lib/python3.8/multiprocessing/pool.py", line 268, in del
File "/home/daemonh/miniconda3/envs/d2l/lib/python3.8/multiprocessing/queues.py", line 362, in put
AttributeError: 'NoneType' object has no attribute 'dumps'

Does it need to focus or just common issue need to ignore, thanks you very much!

Environment

----------Python Info----------
Version : 3.8.13
Compiler : GCC 7.5.0
Build : ('default', 'Mar 28 2022 11:38:47')
Arch : ('64bit', 'ELF')
----------MXNet Info-----------
Version : 1.9.1
Directory : /home/daemonh/miniconda3/envs/d2l/lib/python3.8/site-packages/mxnet
Commit hash file "/home/daemonh/miniconda3/envs/d2l/lib/python3.8/site-packages/mxnet/COMMIT_HASH" not found. Not installed from pre-built package or built from source.
Library : ['/home/daemonh/miniconda3/envs/d2l/lib/python3.8/site-packages/mxnet/libmxnet.so']
Build features:
✔ CUDA
✔ CUDNN
✔ NCCL
✔ CUDA_RTC
✖ TENSORRT
✔ CPU_SSE
✔ CPU_SSE2
✔ CPU_SSE3
✖ CPU_SSE4_1
✖ CPU_SSE4_2
✖ CPU_SSE4A
✖ CPU_AVX
✖ CPU_AVX2
✔ OPENMP
✖ SSE
✖ F16C
✖ JEMALLOC
✔ BLAS_OPEN
✖ BLAS_ATLAS
✖ BLAS_MKL
✖ BLAS_APPLE
✔ LAPACK
✔ MKLDNN
✔ OPENCV
✖ CAFFE
✖ PROFILER
✔ DIST_KVSTORE
✖ CXX14
✖ INT64_TENSOR_SIZE
✔ SIGNAL_HANDLER
✖ DEBUG
✖ TVM_OP
----------System Info----------
Platform : Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.17
system : Linux
release : 5.10.102.1-microsoft-standard-WSL2
version : #1 SMP Wed Mar 2 00:30:59 UTC 2022

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions