Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importing torch crashed when metacall is imported first. #460

Open
iripoll opened this issue Sep 5, 2023 · 2 comments
Open

Importing torch crashed when metacall is imported first. #460

iripoll opened this issue Sep 5, 2023 · 2 comments
Labels
bug Something isn't working c/c++ Pull requests that update C/C++ code python Pull requests that update Python code

Comments

@iripoll
Copy link

iripoll commented Sep 5, 2023

🐛 Bug Report

Importing torch crashed when metacall is imported first.

Expected Behavior

Load torch silently.

Current Behavior

ipython and juputer-notebook crashes when importing torch.

Steps to Reproduce

$ ipython3 
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from metacall import metacall

In [2]: import torch
Segmentation fault (core dumped)

Versions:

  • torch: 2.0.1+cu117
  • MetaCall: 0.7.4

Context (Environment)

Detailed Description

Possible Implementation

@iripoll iripoll added the bug Something isn't working label Sep 5, 2023
@giarve giarve added good first issue Good for newcomers python Pull requests that update Python code labels Sep 5, 2023
@viferga
Copy link
Member

viferga commented Sep 5, 2023

@iripoll Thank you so much for your report.

This is a limitation related to metacall that cannot run outside of metacall. This problem basically happens because once it's run outside metacall, it initializes two instances of python. The solution of this has been investigated and the proposal is to detect if it's being run under an existing runtime and then relink the loaders on the fly to the process instead of loading the loader with the libpython dependency.

Here is the issue described in more detail: #231

The same problem happens here with NodeJS: #31

I have investigated this in python and it happens also due to openssl but in this case is due to different versions of crypto library which have a dependency to openssl.

Our distributable mitigates the issue but it's not ideal, eventually we will solve this completely for all runtimes. We only need a bit of magic.

I am gonna keep the issue open so we can have more reference and cases where this happens for the future.

Meanwhile we also have an integration with Jupyter, anybody needs to run metacall on top of it: https://github.com/metacall/jupyter-kernel

@viferga viferga added c/c++ Pull requests that update C/C++ code and removed good first issue Good for newcomers labels Sep 5, 2023
@ChinmayK0607
Copy link

This happened with me as well when importing metacall on a colab notebook.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working c/c++ Pull requests that update C/C++ code python Pull requests that update Python code
Projects
None yet
Development

No branches or pull requests

4 participants