Skip to content

Commit

Permalink
bpo-43392: Optimize repeated calls to __import__() (pythonGH-24735)
Browse files Browse the repository at this point in the history
Implements a two steps check in `importlib._bootstrap._find_and_load()` to avoid locking when the module has been already imported and it's ready.

---

Using `importlib.__import__()`, after this, does show a big difference:

Before:
```
$ ./python -c 'import timeit; print(timeit.timeit("__import__(\"timeit\")", setup="from importlib import __import__"))'
15.92248619502061
```

After:
```
$ ./python -c 'import timeit; print(timeit.timeit("__import__(\"timeit\")", setup="from importlib import __import__"))'
1.206068897008663
```

---
  • Loading branch information
Kronuz authored Aug 12, 2021
1 parent 953d272 commit 03648a2
Show file tree
Hide file tree
Showing 3 changed files with 539 additions and 517 deletions.
21 changes: 16 additions & 5 deletions Lib/importlib/_bootstrap.py
Original file line number Diff line number Diff line change
Expand Up @@ -1032,17 +1032,28 @@ def _find_and_load_unlocked(name, import_):

def _find_and_load(name, import_):
"""Find and load the module."""
with _ModuleLockManager(name):
module = sys.modules.get(name, _NEEDS_LOADING)
if module is _NEEDS_LOADING:
return _find_and_load_unlocked(name, import_)

# Optimization: we avoid unneeded module locking if the module
# already exists in sys.modules and is fully initialized.
module = sys.modules.get(name, _NEEDS_LOADING)
if (module is _NEEDS_LOADING or
getattr(getattr(module, "__spec__", None), "_initializing", False)):
with _ModuleLockManager(name):
module = sys.modules.get(name, _NEEDS_LOADING)
if module is _NEEDS_LOADING:
return _find_and_load_unlocked(name, import_)

# Optimization: only call _bootstrap._lock_unlock_module() if
# module.__spec__._initializing is True.
# NOTE: because of this, initializing must be set *before*
# putting the new module in sys.modules.
_lock_unlock_module(name)

if module is None:
message = ('import of {} halted; '
'None in sys.modules'.format(name))
raise ModuleNotFoundError(message, name=name)

_lock_unlock_module(name)
return module


Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
:func:`importlib._bootstrap._find_and_load` now implements a two-step
check to avoid locking when modules have been already imported and are
ready. This improves performance of repeated calls to
:func:`importlib.import_module` and :func:`importlib.__import__`.
Loading

0 comments on commit 03648a2

Please sign in to comment.