Wednesday, December 08, 2004
PARALLEL MATLAB METHODS
A. Embarrassingly Parallel
Software that make use of this approach: Multi, Paralize, PLab, Parmatlab. This approach makes use of multiple MATLAB processes running on different machines or a singlemachine with multiple processors. However no coordination between the MATLAB processes is provided..Instead, a parent process passes off data to the child processes. Then all processes work on its local data,and return the result to the parent process.Under this model, the type of functions that can be parallelized is limited. For example, a for-loopwith any data dependency across iteration would be impossible to parallelize under this model. However,this model has the advantage of simplicity. In the software we found that utilize this approach, usuallyno change in existing code is needed to parallelize the code, if the code is parallelizable using thissimple approach. From our experience, this approach is sufficient for a lot of real world applications.
B. Message Passing
Software that make use of this approach: MultiMATLAB, CMTM, DPToolbox, MPITB/PVMTB, MATmarks, PMI, PTToolbox, MatlabMPI, pMatlab.This approach provides message passing routines between MATLAB processes. This enables users towrite their own parallel programs in MATLAB in a fashion similar to writing parallel programs with acompiled language using MPI. In fact, for some of the projects, the routines providedare wrappers for MPI routines. This approach has the advantage of flexibility: users are theoreticallyable to build any parallel system in MATLAB that they can build in compiled languages with MPI. Thisapproach is a superset of the embarrassingly parallel approach.
C. Backend Support
Software that make use of this approach: NetSolve, DLab, Matpar, PLAPACK, PARAMAT, MATLAB*P. This approach uses MATLAB as a front end for a parallel computation engine. Computation is doneon the engine, usually making use of high performance numerical computation libraries like ScaLAPACK. For some approaches, the data resides in MATLAB and are passed to the engine andback. And for some approaches, the data reside on the server and are passed back to MATLABonly upon request. The latter approach has the advantage that there is less data traffic. This is importantfor performance when data sets are large.The advantage of this approach is that it only requires one MATLAB session (therefore only onelicense), and it usually does not require the end user to have knowledge of parallel programming.
D. MATLAB Compilers
Software that make use of this approach: Otter, RTExpress, ParAL, FALCON, CONLAB, MATCH, Menhi. These softwares compile MATLAB scripts into an executable, sometimes translating the scripts into acompiled language as an intermediate step. Some softwares links the compiled code withparallel numerical libraries, while some softwares generates code that is already parallel.This approach has the advantage that the compiled code runs without the overhead incurred byMATLAB. Also the compiled code can be linked with code written in C or FORTRAN, so MATLABcan be used to develop part of a system instead of the whole system. In this approach, MATLAB isused as a development platform instead of a computing environment. This allows the produced parallelprogram to run on platforms which does not support MATLAB (e.g. SGI).
Software that make use of this approach: Multi, Paralize, PLab, Parmatlab. This approach makes use of multiple MATLAB processes running on different machines or a singlemachine with multiple processors. However no coordination between the MATLAB processes is provided..Instead, a parent process passes off data to the child processes. Then all processes work on its local data,and return the result to the parent process.Under this model, the type of functions that can be parallelized is limited. For example, a for-loopwith any data dependency across iteration would be impossible to parallelize under this model. However,this model has the advantage of simplicity. In the software we found that utilize this approach, usuallyno change in existing code is needed to parallelize the code, if the code is parallelizable using thissimple approach. From our experience, this approach is sufficient for a lot of real world applications.
B. Message Passing
Software that make use of this approach: MultiMATLAB, CMTM, DPToolbox, MPITB/PVMTB, MATmarks, PMI, PTToolbox, MatlabMPI, pMatlab.This approach provides message passing routines between MATLAB processes. This enables users towrite their own parallel programs in MATLAB in a fashion similar to writing parallel programs with acompiled language using MPI. In fact, for some of the projects, the routines providedare wrappers for MPI routines. This approach has the advantage of flexibility: users are theoreticallyable to build any parallel system in MATLAB that they can build in compiled languages with MPI. Thisapproach is a superset of the embarrassingly parallel approach.
C. Backend Support
Software that make use of this approach: NetSolve, DLab, Matpar, PLAPACK, PARAMAT, MATLAB*P. This approach uses MATLAB as a front end for a parallel computation engine. Computation is doneon the engine, usually making use of high performance numerical computation libraries like ScaLAPACK. For some approaches, the data resides in MATLAB and are passed to the engine andback. And for some approaches, the data reside on the server and are passed back to MATLABonly upon request. The latter approach has the advantage that there is less data traffic. This is importantfor performance when data sets are large.The advantage of this approach is that it only requires one MATLAB session (therefore only onelicense), and it usually does not require the end user to have knowledge of parallel programming.
D. MATLAB Compilers
Software that make use of this approach: Otter, RTExpress, ParAL, FALCON, CONLAB, MATCH, Menhi. These softwares compile MATLAB scripts into an executable, sometimes translating the scripts into acompiled language as an intermediate step. Some softwares links the compiled code withparallel numerical libraries, while some softwares generates code that is already parallel.This approach has the advantage that the compiled code runs without the overhead incurred byMATLAB. Also the compiled code can be linked with code written in C or FORTRAN, so MATLABcan be used to develop part of a system instead of the whole system. In this approach, MATLAB isused as a development platform instead of a computing environment. This allows the produced parallelprogram to run on platforms which does not support MATLAB (e.g. SGI).
# posted by Micro Struct @ 5:40 PM 
