cppython: bring C++ classes to Python

• How to export C++ class
• Files generated
• How to use
• todo
• How to run test
• Python GIL
  • Cython GIL

cppython brings C++ classes to Python world using cython

• Automatically generate C++ wrapper classes and cython files
• Support pure virtual method
• C++ virtual method could be override in Python sub class

How to export C++ class

In order to export a non-pod C++ class Foo to python:

1. Export forward name Foo in *.pxd file
2. Create a C++ proxy class inheriting from Foo. It also links with a python object
3. Override virtual methods in proxy class to make them overrideable in python
4. Export proxy class and its methods in *.pxd file
5. Wrap Foo proxy class in *.pyx file as a normal Cython class
6. Implement proxy call API for each of virtual methods

The steps are tedious and boring. cppython helps you automation the whole process

Files generated

Given a header file header_file.hpp which contains all symbols need to export to python module foo, cppython will generate following files:

• head_file.pxd: export C++ symbol to cython
• foo.pyx: the main file for foo module, it will generate foo.h and foo.c through cython to build foo module
• foo.pxi: the public API used for proxy call, will be include by foo.pyx
• foo_cppython.hpp: the proxy classes' defination
• foo_cppython.cpp: the proxy classes' implementation
• foo_cppython.pxd: export C++ proxy class to cython
• setup.py: setup file for build python extension

How to use

1. write a header file which include all the symbols you would like to export, including
   • classes
   • POD structs
   • free functions
   • const int
   • enums
   • marco constants
2. run command python cppython.py -t header-file-for-export.hpp -s <C++ source files> -m path/to/module_name
3. after that you could find generated module_name files under path/to. you could review and modify manually
4. run command cd path/to && python setup.py to actually build the python extension module using cython
5. For detail command line argument list, run python cppython.py -h

todo

• wrap static method as static class method in python, right now it is exported as python instance method
• better support for C++ reference
• better support for C array (using view?)
• forward declaration support
• boost.python support
• support customize C++ entities name
• add include path support
• support more than one constructor,right now it only export the first constructor

How to run test

• ubuntu 64 and Mac are supported out of the box
• mock, enum34 and cython need to be installed first
• run ./test.sh

Python GIL

if C extension spawn threads, you must call PyEval_InitThreads at least once in the main thread. What it does is init GIL and aquire it. So that python itself could be thread safe running any python code.

In child thread any time you want to manipulate Python objects, make sure to call pair PyGILState_Ensure(), PyGILState_Release() to get GIL and release it. There API is safe to call recursively.

If you would like to call some non-Python/c blocking code (like Join()), you could use PyEval_SaveThread() PyEval_RestoreThread() pair to release GIL and re get it

In general situations, the C library needs to call PyEval_InitThreads() to gain GIL before spawning any thread that invokes python callbacks. And the callbacks need to be surrounded with PyGILState_Ensure() and PyGILState_Release() to ensure safe execution.

However, if the C library is running within the context of, say, a python C extension, then there are special cases where it is safe to omit GIL manipulation at all if the sub thread execuation is enclosed by the main thread.

Cython GIL

export C/C++ all marked as nogil to tell cython that they are safe to call with nogil

There are 3 ways to mark nogil, they are identical

• cdef extern from "export.hpp" nogil: nogil for all entity inside export.hpp
• cdef cppclass SomeClass(BaseClass) nogil: nogil for all class members
• void some_member_function(const char * arg) nogil except + nogil for a single member

When calling, it could be invoke without gil explicitly:

with nogil:

self._this.register_front(address)

Notice that without with nogil:, the function still called with GIL wheather or not it has been marked as nogil if you call non nogil quanlifier member inside with nogil: scope, cython will report error message

Calling gil-requiring function not allowed without gil

When called with nogil:, the generated code will have PyEval_SaveThread() PyEval_RestoreThread() pair enclosed the call.