1/12/2007

RUN safely on "underground"

Introduction:
Tube is a 34-processor cluster, owned by Dr. Beal and administered by CSE, available to members of his group for algorithm prototyping. Each node has 2GB ram, 80GB hd, and dual core 3.2GHz Intel Xeon processors. underground.cse.buffalo.edu is the head-node, accessible only via ssh. Matlab is available with the Distributed Computing Toolbox.

Tips to follow up:
1. Make sure you can run the script on local machine first. It's hard for us to terminate a bad heavy job on cluster. I have no such authority.
2. Write a LOG file to know where you may get stuck.
3. Save resulting variable with huge size into a FILE, instead of returning it directly. I lost results before by returing a big variable, it's probably caused by nasty Java vm.
4. After creating a folder on cluster, make sure it is writable, or you will not be able to write into files. I use cmd 'chmod 777 foldername'.

How to run on underground?
1. Invoke matlab
>matlab
2. Specify absolute pathes to tell the workers where to load script and write log.
> pppp = {'/home/csgrad/juanli/work/toolkits/vbssm_v3.3.5',...
'/home/csgrad/juanli/work/code'};
3. Enjoy the underground trip
> tic; s = dfeval(@func,num2cell(1),'PathDependencies',pppp);toc;

Examples are available at http://www.cse.buffalo.edu/~juanli/scripts
MATLAB online DCT help