Department of Oceanography
1355 Oxford Street
PO Box 15000
Halifax, NS, B3H 4R2
Processing SOPALE Output
TMM (Thermo-mechanical Model)
old SOPALE documentation
Geodynamics home page
On the scrXm family, the stable, and geodyncomp
This feature has changed, and may change again, as sopale_nested evolves. See the /bin directory of any sopale_nested version. Run the script without parameters to get the user documentation.
sopale_nested generates restart files at periodic intervals, and has the ability to continue a model from them.
In order to restart sopale_nested, you will need
Version 2011-09-06_2.0 and earlier do not require the header files. (The files whose names end in _T00_o)
On occasion, it may be desirable to stop a model before it has completed.
Is It Still Running?
Sometimes, such as when sopale_nested can no longer write output files due to quota issues, or because one of the four processes involved in a one-nest job has been killed, the number-crunching processes remain running at 100%, using cpu cycles, but producing no visible results. Based on recent experience, this happens more often than is desirable.
Locating such errant jobs and/or processes can be done by running a script, and filtering the output for one's username. The output includes the date on which the processes were started. That date can be an indicator of a 'lost' job, if it is more in the past than one would expect for a sopale_nested run.
Since version 2009-07-26_29.0
The command to find the pid of your job is in the file
Mix-And-Match (scrXm family and stable)
It is often desirable to use disc space on one server and the cpu of another. To use the cpu of scrom, and disc space on scrim:
You only have to do Steps 4 and 5 once. If either scrom or scrim
is rebooted, the directory
dg322872 wants to use the cpu of sopale-0-1, but discovers he has no quota there. However he has quota on sopale-0-3 in /state/partition2. What to do?
# ssh to sopale-0-1 ssh -X sopale-0-1 # [optional: start filerunner] /share/apps/fr & # make a place to mount the disc of sopale-0-3 mkdir -p geodyncomp-nodes/sopale-0-3 # mount the disc of sopale-0-3 sshfs sopale-0-3:/ geodyncomp-nodes/sopale-0-3 # cd to the directory he wants to use cd geodyncomp-nodes/sopale-0-3 cd state/ cd partition2 # The first time, he will have to create this directory # mkdir dg322872 # The directory already exists, so he just cd's cd dg322872 mkdir -p testing/sopale_nested cd testing/sopale_nested # get a copy of the sopale_nested input file cp -ip /home/dg322872/testing/sopale_nested/sopale_nested_i.Guilmette-makeAnOcean-P1_DIV-20120608_2.0 . # run sopale_nested (line broken for better formatting) /home/dguptill/software/sopale_nested-2012-06-08_2.1/bin/\ start_sopale_nested-2012-06-08_2.1-ompimod-intel-lib-11.0.074-std_large \ 2 no-email nosync sopale_nested_i.Guilmette-makeAnOcean-P1_DIV-20120608_2.0
Mix-And-Match (scrXm and stable to geodyncomp)
It is possible to use the cpu of scrym, and the disc space on a geodyncomp node. The geodyncomp nodes cannot be seen from the outside world, so a two hop strategy is required. Make the sshfs mounts from the scrXm (or stable) machine to geodyncomp to the geodyncomp node.
# login to scrym cd mkdir -p scrXm/geodyncomp sshfs geodyncomp.ocean.dal.ca:/ scrXm/geodyncomp ssh -X geodyncomp.ocean.dal.ca mkdir -p geodyncomp-nodes/sopale-0-0 sshfs sopale-0-0:/ geodyncomp-nodes/sopale-0-0 # create another terminal on scrym cd scrXm/geodyncomp/geodyncomp-nodes/sopale-0-0/<some directory> run sopale_nested