Month: November 2007

Enabling thread support in SWIG and Python

November 23, 2007 Matt 7 Comments SWIG, Python, Software Construction

I was looking for some days in SWIG documentation how I could release the GIL (Global Interpreter Lock) with SWIG. There were some macros defined in the generated code, but none was used in any place.

In fact, I just had to enable the thread support with an additional argument (-threads) and now every wrapped function releases the GIL before it is called, but that does not satisfy me. Indeed, some of my wrappers must retain the GIL while they are used (see this item). So here are the features that can be used :

  • nothread enables or disables the whole thread lock for a function :
    • %nothread activates the nothread feature
    • %thread disables the feature
    • %clearnothread clears the feature
  • nothreadblock enables or disables the block thread lock for a function :
    • %nothreadblock activates the nothreadblock feature
    • %threadblock disables the feature
    • %clearnothreadblock clears the feature
  • nothreadallow enables or disables the allow thread lock for a function :
    • %nothreadallow activates the nothreadallow feature
    • %threadallow disables the feature
    • %clearnothreadallow clears the feature

When the whole thread lock is enabled, the GIL is locked when entering the C function (with the macro SWIG_PYTHON_THREAD_BEGIN_BLOCK). Then it is released before the call to the function (with SWIG_PYTHON_THREAD_BEGIN_ALLOW), retained after the end (SWIG_PYTHON_THREAD_END_ALLOW) and finally it is released when exiting the function (SWIG_PYTHON_THREAD_END_BLOCK), after all Python result variables are created and/or modified.

Using Scons to create Python modules with Visual Studio 2005

November 12, 2007 Matt No Comments Scons, Software Construction

Starting from Visual Studio 2005, every executable or dynamic library must declare the libraries it uses with a manifest file. This manifest can be embedded in the executable or library, and this is the best way to deal with it.

When using Scons, this embedding does not occur automatically. One has to overload the SharedLibrary builder so that a post-action is made after building the library :

def MSVCSharedLibrary(env, library, sources, **args):
  cat=env.OriginalSharedLibrary(library, sources, **args)
  env.AddPostAction(cat, 'mt.exe -nologo -manifest ${TARGET}.manifest -outputresource:$TARGET;2')
  return cat
 
 env['BUILDERS']['OriginalSharedLibrary'] = env['BUILDERS']['SharedLibrary']
 env['BUILDERS']['SharedLibrary'] = MSVCSharedLibrary

With this method, the embedding is made for every library, which is handy. The same can be done for the Program builder with the line :

  env.AddPostAction(cat, 'mt.exe -nologo -manifest ${TARGET}.manifest -outputresource:$TARGET;1')

Wrapping a C++ container in Python

November 6, 2007 Matt 1 Comment Python, C++, SWIG

When moving to Python, the real big problem that arises is the transformation of a Python array into the C++ container the team used for years.

Let’s set some hypothesis :

  • there is a separation between the class containing the data and the class that uses the data (iterators, …)
  • the containing class can be changed (policy or strategy pattern)

The first hypothesis is derived from the responsibility principle, the two classes have two distinct responsibilities, the first allocates the data space and allows simple access to it, the second allows usual operations (assignation, comparison tests or iterations for instance).

The second one will be the heart of the wrapper. It allows to change the way data is stored and accessed in a simple way.

Read More