Twenty Years of Parallel MATLAB

I have just returned from the MathWorks company meeting celebrating our 40th Anniversary. In one of the presentations, Jos Martin described how Parallel MATLAB was introduced almost twenty years ago. Here are a few slides from Jos's talk.

Why There Wasn't Any Parallel MATLAB

In MATLAB News and Notes for spring 1995, I wrote a one-page Cleve's Corner titled "Why there isn't any parallel MATLAB." There were three reasons.

Memory model. MATLAB would generate a matrix on a host machine, split it into roughly equally sized submatrices, and distribute each submatrix to a node. But it took far longer to distribute the data then it did to do the computation. Any matrix that would fit into memory on the host was too small to make effective use of the parallel computer itself.

Granularity. The amount of work involved in a single matrix computation is too little to be effectively parallelized.

Business situation. There are too few potential customers at this time (1995) to undertake fundamental changes in MATLAB's architecture.

This one-page note turned out to be one of my most widely cited publications.

Twenty-seven Parallel MATLABs

A 2001 survey by Ron Choy at MIT found 27 different projects that were developing some way to run MATLAB in parallel. All of them involved a MATLAB-based host program calling a fixed library of parallel functions, written in some other language, on the workers. None of the systems were capable of running arbitrary MATLAB programs in parallel. None of them were MathWorks products.

Distributed Computing Toolbox

MathWorks introduced the MATLAB Distributed Computing Toolbox in November 2004. We improvised this demo setup at our first Supercomputing Conference, SC2004 in Pittsburg,

Supercomputing Conference

A year later, SC2005 was in Seattle and our booth featured four worker machines on a wire shelving unit purchased at a local Home Depot.

Bill Gates

Since Seattle was his home town, Bill Gates gave the keynote talk at SC2005. He announced that Microsoft was going into High Performance Computing and used the MathWorks Distributed Computing Toolbox in his demonstration.

Now There is a Parallel MATLAB

So, a little more than ten years after the first Cleve's Corner about parallel computing, a second Cleve's Corner in News and Notes was able to reverse the situation.

Get the MATLAB code

Published with MATLAB® R2023a

hpc.social

Contents