What an appetizing title! This book is part of an O’Reilly serie that treats a lot of interesting topic. Contrary to Beautiful Code, this one is much shorter but the title suggest it is much more pragmatic.
Continue reading Book review: The Productive Programmer
Since this post, Intel has officially released Parallel Studio. This is why I’ve published a new, up-to-date review here.
Profiling comes in three different flaviors. The first is emulation, where a processor behavior is emulated, the second is sampling, where at regular intervals, the profiler samples the status of a program, and fianlly instrulentation, where the profiler gets information when a subroutine is called and when it returns. As with the Heisenberg uncertainty, profiling changes the exact behavior of your program. This is something you have to remember when analyzing a profile.
Valgrind is an Open Source emulation profiler. It is freely available on standard Linux platforms. As it is an emulation, it is far slower than the actual program. This means that the I/O are underestimated. The advantage is that you can have every detail on the memory behavior (cache misses for instance). Valgrind does not emulate all processors, but you can tweak it to approach your own one.
Continue reading Profiling with Valgrind/Callgrind
Some months ago, I had a TotalView tutorial, thanks to my job. Now, I’ve actually used it to debug one of my parallel applications and I would like to share my experience with fantastic tool.
First TotalView is not only a parallel debugger available on several Linux and Unix platforms. It also is a memory checker (MemoryScape and the TotalView plugin) as well as a reverse debugger, that is, you can roll back the execution of a program, even after it crashed (where it would be useless with a standard debugger like GDB).
Continue reading Overview of TotalView, a parallel debugger
Thanks to the Numpy mailing list (more specially Stéfan van der Waltz and David Cournapeau), I’ve found a new Project Management Software, Redmine. Now, I needed to set it up without an access to the Internet, and as I didn’t have ana ccess to a MySQL server, I used SQLite as a DB backend.
Continue reading Setting up a Redmine application on a specific location
I’m a very curious guy, and I wanted to know who is looking at my blog, and for my wife, who is interested by what is viewed on her decoration site (in construction as she wants to make a living of decoration advice). With my hosting service, I have access to Awstats, but Google Analytics seems better suited for data analysis. And this is what this book explains.
Continue reading Book review: Google Analytics 2.0
I chose Eclipse as my new Linux IDE, instead of Konqueror + KWrite. The purpose was to be able to launch a SCons build from the IDE, get the errors in a panel and double-clicking on one of them would direct me to the location of the error.
So Eclipse seemed to fit my needs:
- Plug-ins to add the support of various languages
- Support of different construction tools
- Support from the main C/C++/Fortran compiler developers (GNU, Intel, IBM, …)
So I will know show you two ways of enabling SCons support for Eclipse.
Continue reading Using SCons with Eclipse (Linux)
As I have to parellize some programs developed in my new lab, I monitor CPU usage during thier execution. I do not usually need MPI to optimize them (although sometimes it is needed), only OpenMP, which means I can track /proc/ to get CPU and instantaneously memory usages.
So I wrote a small script that can be used by anyone for this purpose. I’ll explain how it works now.
Continue reading Monitoring CPU usage in multithreaded applications
Some times ago, I proposed an optional build for SWIG if the SWIG binary was not found on the system. Here I propose an enhancement, a new library builder that will be registered in the environment env as PythonModule. It takes the same arguments as a classical SharedLibrary, but it does some additional steps :
- It forces SWIG to create a Python wrapper (flag -python)
- It checks if SWIG is present at all
- It suppresses every prefix that the system might need (as lib in Linux)
- On Windows and for Python >= 2.5, it changes the extension as pyd
Continue reading Creating a Python module with Scons and SWIG
I was looking for some days in SWIG documentation how I could release the GIL (Global Interpreter Lock) with SWIG. There were some macros defined in the generated code, but none was used in any place.
In fact, I just had to enable the thread support with an additional argument (-threads) and now every wrapped function releases the GIL before it is called, but that does not satisfy me. Indeed, some of my wrappers must retain the GIL while they are used (see this item). So here are the features that can be used :
- nothread enables or disables the whole thread lock for a function :
- %nothread activates the nothread feature
- %thread disables the feature
- %clearnothread clears the feature
- nothreadblock enables or disables the block thread lock for a function :
- %nothreadblock activates the nothreadblock feature
- %threadblock disables the feature
- %clearnothreadblock clears the feature
- nothreadallow enables or disables the allow thread lock for a function :
- %nothreadallow activates the nothreadallow feature
- %threadallow disables the feature
- %clearnothreadallow clears the feature
When the whole thread lock is enabled, the GIL is locked when entering the C function (with the macro SWIG_PYTHON_THREAD_BEGIN_BLOCK). Then it is released before the call to the function (with SWIG_PYTHON_THREAD_BEGIN_ALLOW), retained after the end (SWIG_PYTHON_THREAD_END_ALLOW) and finally it is released when exiting the function (SWIG_PYTHON_THREAD_END_BLOCK), after all Python result variables are created and/or modified.