Using the MATLAB Distributed/Parallel Computing Toolbox without modifying your cluster

October 2, 2013

This post shows how I set up MATLAB to perform parallel calculations on a multi-core server which is part of an existing cluster that is managed using Torque and Moab. I did this without making any changes to the cluster submission system. The installation documentation for the MATLAB Parallel Computing Toolbox and Distributed Computing Server is poor. The instructions make a lot of assumptions that don’t fit into an existing “production” cluster environment. For example, they assume that a GUI is available to configure the cluster (this step is not required for my method!) Further, the instructions give the impression that MATLAB components must be “installed” on both the head node and compute nodes by an administrator. THIS IS FALSE! Ordinary users can run concurrent (multicore) MATLAB jobs without administrative privileges!

The following method also allows you to run MATLAB with more than 8 or 12 cores-in fact, you can use 16 cores or more on a single machine! It also may be possible to distribute tasks across multiple nodes (hosts), but that’s for an upcoming post. See my repository of PBS submit script examples on GitHub.

Customize the script that launches the MATLAB Distributed Computing Server

Make a copy of the script

MATLAB/R2011a/toolbox/distcomp/bin/mdce_def.sh

and modify it to suite your cluster. In my case, I only had to set the variables PIDBASE, LOCKBASE, LOGBASE, and CHECKPOINTBASE to point to locations in the /tmp directory of each compute node so that ordinary users can write files to those locations. My full script will appear on GitHub.

Create a submit script that runs MATLAB

I created a Torque submit script that automatically starts a MATLAB DCE server, a job manager, and a set of worker processes on the node. The full script will be available on GitHub; I will only highlight a few snippets below. First, start the MATLAB distributed computing server (mdce) on the compute node using the launch script that I created:

export MATLABPARTEMP=/tmp/tmp$PBS_JOBID
mdce start -clean -mdcedef /apps/CompVis/MATLAB/R2011a/toolbox/distcomp/bin/mdce_def_STOKES.sh

Note that I have exported an environment variable that points to a directory within /tmp that will hold all our temporary files; we will clean up this directory when the job is done. My custom mdce launch script needs this environment variable. Next, start the job manager on the compute node:

startjobmanager -clean -name MyJobManager -v

Start worker daemons on the compute node:

for (( i=1; i<=$NP; i++)); do
    echo "Starting worker process $i";
    startworker -jobmanagerhost localhost -jobmanager MyJobManager -name worker$i;
done

Now, start MATLAB and run your code. When the job is done, we need to shut down all the daemons in reverse order:

## Cleanup
sleep 3
for (( i=1; i<=$NP; i++)); do
    echo "Stopping worker process $i";
    stopworker -clean -name worker$i;
done
stopjobmanager -clean -name MyJobManager
mdce stop -clean

Finally, let’s clean up the /tmp directory on the compute node:

rm -rfv $MATLABPARTEMP

Optional: Run MATLAB interactively

If you have used qsub -I … to obtain an interactive session on a compute node, you may manually run the commands above and run parallel commands in an interactive MATLAB session on the compute node. This approach is very helpful for debugging!

Comments

Comment #1 by Dario Paccagnan

Dario Paccagnan - Dec 2, 2013

Hello Craig, your post has been really useful as I need to interface matlab and torque… Is it possible to have the source codes you used?

Comment #2 by Craig

Craig - Dec 3, 2013

I created a repo on Github for example PBS scripts, and added this MATLAB example: https://github.com/cfinch/PBS_scripts Thanks for reminding me to post the code!

Comment #3 by dinesh

dinesh - Mar 3, 2014

carig i need how to create a torque distribution function in matlab using tool box.

Comment #4 by Craig

Craig - Mar 4, 2014

Dinesh, I don’t understand your question.

Comment #5 by dinesh

dinesh - Mar 4, 2014

craig,I am doing project in switched reluctance motor.To reduce the torque ripple by using neural network,for that i need torque distributed function for dividing two input,so i need torque distribution function as toolbox in matlab.

Comment #6 by Craig

Craig - Mar 5, 2014

Dinesh, this post is not about mechanical torque. This post is about a software tool called Torque, which has nothing to do with the torque of a motor.

Comment #7 by Jay

Jay - Jun 5, 2016

Hi Craig, Thank you for this post, I found the information very useful. I am curious about how the cluster profile was configured in this setup? Best, Jay

Comment #8 by Craig

Craig - Jun 0, 2016

Jay, I don’t remember because I haven’t worked on this in several years. I vaguely recall that you actually might not need a cluster profile at all. I suggest that you try running the example in my GitHub repo (https://github.com/cfinch/PBS_scripts/tree/master/matlab-concurrent-hello-world) and see if it works without one.