This is my first review. I read this book some time ago but I still want to write about it because the topic is very interesting.

Content and opinions

The main matter of this book is not about how to write parallel applications, but more on what underlying architecture is interesting for those applications.

First, the reader is introduced with some vocabulary about parallelism (SIMD, MIMD, …) and their impact on applications. Then, interconnections between processors is introduced. In every current parallel application, communication between processors has a big impact at different scale: when more processors are added to the cluster, how the application is distributed, … Several chapters are discussing the issues that users will face.

Once the processors are connected, the different process that are executed need to use a way of communicating: they share some memory or they post messages to each other. Each model has its advantages and drawbacks. They are described in two complete chapters.

These architectures solve problems. The reader is introduced with the concept of abstract model. The underlying assumption is that problem that cannot be parallelly solved on the abstract model cannot be solved with shared memory or message passing architecture. This part is also abstract in its content and is perhaps the hardest part to understand in the book. I agree that mastering the abstract model will ease the pain of design the application, because it would have been split into parallel chunks first, but there is some overhead with using this model.

A complete chapter is dedicated to the Message Passing Interface, MPI. This is a standard that is widely used, and this chapter is perhaps the closest to programming issues (there is some code !). The C implementation is exposed (not the C++ or Fortran’s interface) in a reference-manual way.

The last chapter is about scheduling. This is a very difficult topic that is not introduced in a lot of books. It is a difficult field (an NP-complete problem), and this is hidden in a lot of explanations. Here, the reader has a complete introduction on the topic and can benefit from his reading when he has a lot of paralel task to do.

Conclusion

The topics covered by this book are very pragmatic. Each programmer should read it so that he can understand the issues with communication in a parallel application. The main reason why parallel applications are not so widely spread is : hardware intercommunication does not scale very well, but solutions are finally starting to be available. As a conclusion, I think this book should be in every university library, not on your own one, because it is still very expensive.