Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors caused by code serialization under the Windows system #5

Open
Kidand opened this issue Dec 25, 2023 · 0 comments
Open

Errors caused by code serialization under the Windows system #5

Kidand opened this issue Dec 25, 2023 · 0 comments

Comments

@Kidand
Copy link

Kidand commented Dec 25, 2023

After executing the command line: python train.py demo/pctnet_pretrain.cfg, the following error is reported:

Traceback (most recent call last):
  File "C:\Solidus\git\project\MIS-FM\train.py", line 46, in <module>
    main()
  File "C:\Solidus\git\project\MIS-FM\train.py", line 43, in main
    agent.run()
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\site-packages\pymic\net_run\agent_abstract.py", line 314, in run
    self.train_valid()
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\site-packages\pymic\net_run\agent_seg.py", line 306, in train_valid
    self.trainIter  = iter(self.train_loader)
                      ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\site-packages\torch\utils\data\dataloader.py", line 438, in __iter__
    return self._get_iterator()
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\site-packages\torch\utils\data\dataloader.py", line 386, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\site-packages\torch\utils\data\dataloader.py", line 1039, in __init__
    w.start()
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
                  ^^^^^^^^^^^^^^^^^
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\multiprocessing\context.py", line 336, in _Popen
    return Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\multiprocessing\popen_spawn_win32.py", line 95, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'NetRunAgent.create_dataset.<locals>.worker_init_fn'

(misfm_env) C:\Solidus\git\project\MIS-FM>Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\multiprocessing\spawn.py", line 122, in spawn_main
    exitcode = _main(fd, parent_sentinel)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\82031\anaconda3\envs\misfm_env\Lib\multiprocessing\spawn.py", line 132, in _main
    self = reduction.pickle.load(from_parent)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
EOFError: Ran out of input

This error message indicates that there was a problem when using the multiprocessing module in Python for multi-process data loading. The crux of the error is that multiprocessing cannot serialize (pickle) a locally defined function (here it is worker_init_fn). What would be a reasonable solution?

@Kidand Kidand changed the title windows系统下代码的序列化导致的报错 Errors caused by code serialization under the Windows system Dec 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant