Skip to content

Commit

Permalink
Much more efficient generation of function signatures, updated docs
Browse files Browse the repository at this point in the history
This modification taps into some newer C++14 features (if present) to
generate function signatures considerably more efficiently at compile
time rather than at run time.

With this change, pybind11 binaries are now *2.1 times* smaller compared
to the Boost.Python baseline in the benchmark. Compilation times get a
nice improvement as well.

Visual Studio 2015 unfortunately doesn't implement 'constexpr' well
enough yet to support this change and uses a runtime fallback.
  • Loading branch information
Wenzel Jakob committed Jan 17, 2016
1 parent 2ac5044 commit 66c9a40
Show file tree
Hide file tree
Showing 19 changed files with 731 additions and 560 deletions.
21 changes: 15 additions & 6 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,16 @@ endif()
find_package(PythonInterp ${PYTHONLIBS_VERSION_STRING} EXACT REQUIRED)

if (CMAKE_CXX_COMPILER_ID MATCHES "Clang" OR CMAKE_CXX_COMPILER_ID MATCHES "GNU")
# Enable C++11 mode on C++ / Clang
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
CHECK_CXX_COMPILER_FLAG("-std=c++14" HAS_CPP14_FLAG)
CHECK_CXX_COMPILER_FLAG("-std=c++11" HAS_CPP11_FLAG)

if (HAS_CPP14_FLAG)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14")
elseif (HAS_CPP11_FLAG)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
else()
message(FATAL_ERROR "Unsupported compiler -- pybind11 requires C++11 support!")
endif()

# Enable link time optimization and set the default symbol
# visibility to hidden (very important to obtain small binaries)
Expand Down Expand Up @@ -74,14 +82,15 @@ include_directories(include)
set(PYBIND11_HEADERS
include/pybind11/cast.h
include/pybind11/common.h
include/pybind11/complex.h
include/pybind11/descr.h
include/pybind11/functional.h
include/pybind11/numpy.h
include/pybind11/operators.h
include/pybind11/pybind11.h
include/pybind11/pytypes.h
include/pybind11/typeid.h
include/pybind11/numpy.h
include/pybind11/complex.h
include/pybind11/stl.h
include/pybind11/functional.h
include/pybind11/typeid.h
)

# Create the binding library
Expand Down
17 changes: 12 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@ become an excessively large and unnecessary dependency.

Think of this library as a tiny self-contained version of Boost.Python with
everything stripped away that isn't relevant for binding generation. The core
header files only require ~2K lines of code and depend on Python (2.7 or 3.x)
header files only require ~3K lines of code and depend on Python (2.7 or 3.x)
and the C++ standard library. This compact implementation was possible thanks
to some of the new C++11 language features (tuples, lambda functions and
variadic templates). Since its creation, this library has grown beyond
Boost.Python in many ways, leading to dramatically simpler binding code in many
common situations.
to some of the new C++11 language features (specifically: tuples, lambda
functions and variadic templates). Since its creation, this library has grown
beyond Boost.Python in many ways, leading to dramatically simpler binding code
in many common situations.

Tutorial and reference documentation is provided at
[http://pybind11.readthedocs.org/en/latest](http://pybind11.readthedocs.org/en/latest).
Expand Down Expand Up @@ -71,6 +71,13 @@ In addition to the core functionality, pybind11 provides some extra goodies:
- Everything is contained in just a few header files; there is no need to link
against any additional libraries.

- Binaries are generally smaller by a factor of 2 or more compared to
equivalent bindings generated by Boost.Python.

- When supported by the compiler, two new C++14 features (relaxed constexpr,
return value deduction) such as are used to do additional work at compile
time, leading to smaller binaries.

### License

pybind11 is provided under a BSD-style license that can be found in the
Expand Down
2 changes: 1 addition & 1 deletion docs/benchmark.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ def generate_dummy_code_boost(nclasses=10):
f.write(codegen(nclasses))
n1 = dt.datetime.now()
os.system("g++ -Os -shared -rdynamic -undefined dynamic_lookup "
"-fvisibility=hidden -std=c++11 test.cpp -I include "
"-fvisibility=hidden -std=c++14 test.cpp -I include "
"-I /System/Library/Frameworks/Python.framework/Headers -o test.so")
n2 = dt.datetime.now()
elapsed = (n2 - n1).total_seconds()
Expand Down
43 changes: 28 additions & 15 deletions docs/benchmark.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,12 @@ Benchmark
The following is the result of a synthetic benchmark comparing both compilation
time and module size of pybind11 against Boost.Python.

A python script (see the ``docs/benchmark.py`` file) was used to generate a
set of dummy classes whose count increases for each successive benchmark
(between 1 and 512 classes in powers of two). Each class has four methods with
Setup
-----

A python script (see the ``docs/benchmark.py`` file) was used to generate a set
of files with dummy classes whose count increases for each successive benchmark
(between 1 and 2048 classes in powers of two). Each class has four methods with
a randomly generated signature with a return value and four arguments. (There
was no particular reason for this setup other than the desire to generate many
unique function signatures whose count could be controlled in a simple way.)
Expand Down Expand Up @@ -43,28 +46,38 @@ compilation was done with

.. code-block:: bash
Apple LLVM version 7.0.0 (clang-700.0.72)
Apple LLVM version 7.0.2 (clang-700.1.81)
and the following compilation flags

.. code-block:: bash
g++ -Os -shared -rdynamic -undefined dynamic_lookup -fvisibility=hidden -std=c++11
g++ -Os -shared -rdynamic -undefined dynamic_lookup -fvisibility=hidden -std=c++14
Compilation time
----------------

The following log-log plot shows how the compilation time grows for an
increasing number of class and function declarations. pybind11 includes fewer
headers, which initially leads to shorter compilation times, but the
performance is ultimately very similar (pybind11 is 1 second faster for the
largest file, which is less than 1% of the total compilation time).
increasing number of class and function declarations. pybind11 includes many
fewer headers, which initially leads to shorter compilation times, but the
performance is ultimately fairly similar (pybind11 is 19.8 seconds faster for
the largest largest file with 2048 classes and a total of 8192 methods -- a
modest **1.2x** speedup relative to Boost.Python, which required 116.35
seconds).

.. image:: pybind11_vs_boost_python1.svg

Differences between the two libraries become more pronounced when considering
the file size of the generated Python plugin. Note that the plot below does not
include the size of the Boost.Python shared library, hence Boost actually has a
slight advantage.
Module size
-----------

Differences between the two libraries become much more pronounced when
considering the file size of the generated Python plugin: for the largest file,
the binary generated by Boost.Python required 16.8 MiB, which was **2.17
times** / **9.1 megabytes** larger than the output generated by pybind11. For
very small inputs, Boost.Python has an edge in the plot below -- however, note
that it stores many definitions in an external library, whose size was not
included here, hence the comparison is slightly shifted in Boost.Python's
favor.

.. image:: pybind11_vs_boost_python2.svg

Despite this, the libraries procuced by Boost.Python for more than a few
functions are consistently larger by a factor of 1.75.
12 changes: 10 additions & 2 deletions docs/cmake.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,16 @@ and that the pybind11 repository is located in a subdirectory named :file:`pybin
# find_package(PythonInterp ${PYTHONLIBS_VERSION_STRING} EXACT REQUIRED)
if (CMAKE_CXX_COMPILER_ID MATCHES "Clang" OR CMAKE_CXX_COMPILER_ID MATCHES "GNU")
# Enable C++11 mode on C++ / Clang
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
CHECK_CXX_COMPILER_FLAG("-std=c++14" HAS_CPP14_FLAG)
CHECK_CXX_COMPILER_FLAG("-std=c++11" HAS_CPP11_FLAG)
if (HAS_CPP14_FLAG)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14")
elseif (HAS_CPP11_FLAG)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
else()
message(FATAL_ERROR "Unsupported compiler -- at least C++11 support is needed!")
endif()
# Enable link time optimization and set the default symbol
# visibility to hidden (very important to obtain small binaries)
Expand Down
17 changes: 12 additions & 5 deletions docs/intro.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@ become an excessively large and unnecessary dependency.

Think of this library as a tiny self-contained version of Boost.Python with
everything stripped away that isn't relevant for binding generation. The core
header files only require ~2K lines of code and depend on Python (2.7 or 3.x)
header files only require ~3K lines of code and depend on Python (2.7 or 3.x)
and the C++ standard library. This compact implementation was possible thanks
to some of the new C++11 language features (tuples, lambda functions and
variadic templates). Since its creation, this library has grown beyond
Boost.Python in many ways, leading to dramatically simpler binding code in many
common situations.
to some of the new C++11 language features (specifically: tuples, lambda
functions and variadic templates). Since its creation, this library has grown
beyond Boost.Python in many ways, leading to dramatically simpler binding code
in many common situations.

Core features
*************
Expand Down Expand Up @@ -64,3 +64,10 @@ In addition to the core functionality, pybind11 provides some extra goodies:

- Everything is contained in just a few header files; there is no need to link
against any additional libraries.

- Binaries are generally smaller by a factor of 2 or more compared to
equivalent bindings generated by Boost.Python.

- When supported by the compiler, two new C++14 features (relaxed constexpr,
return value deduction) such as are used to do additional work at compile
time, leading to smaller binaries.
Loading

0 comments on commit 66c9a40

Please sign in to comment.