Python

Getting Started with Pybind11

Jake @Scicoding

May 9, 2023 • 4 min read

Pybind11 is a lightweight header-only library that provides a means to create Python bindings for C++ code. It is a popular choice for exposing C++ classes, functions, and other components to Python, enabling seamless interaction between the two languages. Pybind11 makes it easy to create Python extension modules using C++11 features, such as lambda functions, smart pointers, and automatic type conversions.

The main goal of Pybind11 is to provide a user-friendly, efficient, and flexible interface between Python and C++ while minimizing the overhead associated with language bindings. It is often used to increase the performance of computationally intensive Python programs by offloading heavy computation to C++ code or to wrap existing C++ libraries for use in Python.

To use Pybind11, developers need to include its headers and write C++ code that defines the bindings between Python and C++ components. After compiling the C++ code into a shared library, the resulting extension module can be imported and used in Python just like any other Python module.

Applications in numerical computation

Pybind11 is helpful for numerical and scientific computations for several reasons:

Performance improvement: Python, an interpreted language, can be slow for certain computational tasks, especially those involving heavy numerical calculations. C++ is a compiled language with faster execution speeds. By using Pybind11 to create Python bindings for C++ code, you can leverage the performance benefits of C++ in your Python applications, leading to significant performance improvements.
Utilizing existing C++ libraries: There are a wealth of high-quality, highly optimized numerical and scientific libraries written in C++ (such as Eigen, Boost, Armadillo, etc.). Pybind11 allows you to create Python bindings for these libraries, making it easy to access their functionality from Python without having to rewrite or reimplement them in Python.
Seamless integration: Pybind11 provides an easy-to-use interface between Python and C++ that allows for seamless integration of C++ code with Python libraries like NumPy, SciPy, and others. This enables you to combine the best of both worlds: the ease of use and vast ecosystem of Python, and the performance and optimization of C++.
Automatic type conversions: Pybind11 supports automatic type conversions between Python and C++ types, making it easier to work with numerical data across the two languages. For example, you can pass NumPy arrays directly to C++ functions and have them automatically converted to corresponding C++ types (e.g., Eigen matrices). This simplifies the process of working with numerical data and minimizes the need for manual type conversions.
Parallelism and concurrency: C++ offers various tools and libraries for parallel and concurrent programming, such as OpenMP, CUDA, and thread support in the C++ Standard Library. Exposing C++ functions to Python through Pybind11 allows you to use these capabilities to parallelize or optimize your numerical computations, leading to further performance improvements.

By incorporating Pybind11 in your numerical and scientific computing projects, you can harness the power of C++ while maintaining the ease of use and flexibility provided by the Python ecosystem.

Installing Pybind11

Setting up Pybind11 is a straightforward process. It can be found in the Python package index PyPi. Thus, installing it with pip is simple

pip install pybind11

PyBind11 installation with pip

Pybind11 is now ready to use.

Example C++ extension using CMake

The next step is to write the C++ code that you want to expose to Python. For convenience, we will use CMake to build our extension.

Before diving into the actual implementation, we need to install the CMake and Pybind11 development package

sudo apt install cmake pybind11-dev

Installing CMake and PyBind11 development package

Let's create a simple math_functions.cpp file that contains a add function:

#include <pybind11/pybind11.h>

int add(int a, int b) {
    return a + b;
}

namespace py = pybind11;

PYBIND11_MODULE(math_functions, m) {
    m.doc() = "pybind11 example plugin"; // Optional module docstring
    m.def("add", &add, "A function that adds two numbers");
}

Add function for PyBind11

The PYBIND11_MODULE macro creates a Python module named math_functions and registers the add function.

First, create a CMakeLists.txt file in the same directory as your math_functions.cpp:

cmake_minimum_required(VERSION 3.4)
project(math_functions)

find_package(pybind11 REQUIRED)

pybind11_add_module(math_functions math_functions.cpp)

CMake template for Pybind11

Now, create a build directory and run CMake:

mkdir build
cd build
cmake ..
make

Building Pybind11 extension module

This will generate a shared library file named math_functions.*.so (or math_functions.*.pyd on Windows) in the build directory.

Using the extension module in Python

Now you can use your compiled extension module in Python:

import math_functions

result = math_functions.add(3, 4)
print(f"3 + 4 = {result}")

Python test code for Pybind11 extension module

This should output: 3 + 4 = 7

That's it! You've successfully set up Pybind11 and created a Python extension module using C++ code. You can now create more complex C++ functions and classes and expose them to Python using Pybind11.

Applications in numerical computation

Installing Pybind11

Example C++ extension using CMake

Using the extension module in Python

Further reading