死亡Error:OSError: [Errno 12] Cannot allocate memory
调试背景:使用的是github上https://github.com/arunmallya/packnet这里的代码。
调试的时候,出现Error,如下:
main()
File "main.py", line 378, in main
manager.prune()
File "main.py", line 263, in prune
savename='_final', best_accuracy=accuracy)
File "main.py", line 217, in train
self.do_epoch(epoch_idx, optimizer)
File "main.py", line 174, in do_epoch
for batch, label in tqdm(self.train_data_loader, desc='Epoch: %d ' % (epoch_idx)):
File "/home/rvlg/anaconda3/envs/torch/lib/python3.5/site-packages/tqdm/_tqdm.py", line 1032, in __iter__
for obj in iterable:
File "/home/rvlg/anaconda3/envs/torch/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 301, in __iter__
return DataLoaderIter(self)
File "/home/rvlg/anaconda3/envs/torch/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 158, in __init__
w.start()
File "/home/rvlg/anaconda3/envs/torch/lib/python3.5/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/rvlg/anaconda3/envs/torch/lib/python3.5/multiprocessing/context.py", line 212, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/rvlg/anaconda3/envs/torch/lib/python3.5/multiprocessing/context.py", line 267, in _Popen
return Popen(process_obj)
File "/home/rvlg/anaconda3/envs/torch/lib/python3.5/multiprocessing/popen_fork.py", line 20, in __init__
self._launch(process_obj)
File "/home/rvlg/anaconda3/envs/torch/lib/python3.5/multiprocessing/popen_fork.py", line 67, in _launch
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
遇到这个问题,由于代码本身的额原因先是考虑到运行电脑的内存问题,于是用
watch -n 2 nvidia-smi
watch -n 2 free -m
全程监视电脑CPU、GPU,以及物理内存、交换区内存的变化情况,发现并不是内存的原因。找bug未果。
换了一个思路,从出错的代码以及错误提示上来看,是dataloader.py出了问题,于是Google,关键词:dataloader OSError: [Errno 12] Cannot allocate memory
果然有很多人也是由于在dataload的时候出错,找了很多原因:
1、电脑内存原因(已排除)
2、电脑系统线程数量限制:https://blog.csdn.net/m0_37644085/article/details/92795488:修改最大进程数(尝试无效)
3、设置pin_memory=False;(尝试无效)
4、修改多线程数量:设置num_workers,系统默认的数量是4,改成1之后,没有效果,后面改成0,问题解决!!!程序可以跑了。
特发此帖纪念,认真查了两天多!!!希望可以帮到大家。