Dalhousie Geodynamics Group

Department of Oceanography
Dalhousie University
1355 Oxford Street
PO Box 15000
Halifax, NS, B3H 4R2


Sopale Nested

Processing SOPALE Output



TMM (Thermo-mechanical Model)

old SOPALE documentation

Geodynamics home page


On the scrXm family, the stable, and geodyncomp

  • ssh to a sopale-enabled computer.
  • Change to the directory where you want to do this run.
  • Prepare an input file, sometimes called sopale_nested_i, using your favourite text editor. Later versions of sopale_nested will accept the name of the input file as a parameter. For those versions, any filename will do.
  • Select a version of sopale_nested from:
    • ls -ld /home/dguptill/software/sopale_nested*
  • Select a build from:
    • ls -l /home/dguptill/software/sopale_nested-<version>/bin
  • Start it, using the 'start' script for the build size you have selected:
  • The script usually requires parameters. When invoked without parameters - as in the example above - it will print a message describing the parameters required.


This feature has changed, and may change again, as sopale_nested evolves. See the /bin directory of any sopale_nested version. Run the script without parameters to get the user documentation.

  • ls -l /home/dguptill/software/sopale_nested-<version>/bin


sopale_nested generates restart files at periodic intervals, and has the ability to continue a model from them.

In order to restart sopale_nested, you will need

  1. an input file for sopale_nested. The input file from the original run can be used, with the following changes:
    • "rres" needs to be set to 1.
    • If "align_time" = 1 then the model will restart at the timestep and time in the restart files.
    • If "align_time" = 0 then the model will restart with timestep = 1, and time = 0.
    • Change "nameout1" to match "nameout1" in the restart file names.

  2. restart files. To make the restart files:
    • rename any matching set of output files
      1. <nameout1>g01_p00_T<ff>_o
      2. <nameout1>g01_p00_f<ff>_o
      3. <nameout1>g01_p0n_T<ff>_o (one of these for each nest, where 'n' varies from 1 up)
      4. <nameout1>g01_p0n_f<ff>_o (one of these for each nest, where 'n' varies from 1 up)
      5. <nameout1>g02_p00_T<ff>_o
      6. <nameout1>g02_p00_f<ff>_o
      by changing
      • <nameout1> to the new run_name
      • <ff> to 00
      • '_o' to '_i'
    • There is a script which automates these name changes. Look for 'make_restart_files' in

Version 2011-09-06_2.0 and earlier do not require the header files. (The files whose names end in _T00_o)


On occasion, it may be desirable to stop a model before it has completed.

Is It Still Running?

Sometimes, such as when sopale_nested can no longer write output files due to quota issues, or because one of the four processes involved in a one-nest job has been killed, the number-crunching processes remain running at 100%, using cpu cycles, but producing no visible results. Based on recent experience, this happens more often than is desirable.

Locating such errant jobs and/or processes can be done by running a script, and filtering the output for one's username. The output includes the date on which the processes were started. That date can be an indicator of a 'lost' job, if it is more in the past than one would expect for a sopale_nested run.

~dguptill/software/bin/list-all-running-sopale_nested | grep dg322872

Since version 2009-07-26_29.0

The command to find the pid of your job is in the file

  • <nameout1>.out
as well as in the email, if you asked for an email when starting sopale_nested. Once you know the pid, stop it with
  • kill <pid>

Mix-And-Match (scrXm family and stable)

It is often desirable to use disc space on one server and the cpu of another. To use the cpu of scrom, and disc space on scrim:

  1. login to scrom
  2. cd
  3. fr &
  4. mkdir -p scrXm/scrim
  5. sshfs scrim.ocean.dal.ca:/ scrXm/scrim
  6. cd scrXm/scrim/<some directory>
  7. run sopale_nested

You only have to do Steps 4 and 5 once. If either scrom or scrim is rebooted, the directory scrXm/scrim will be empty. Repeat step 5 to fill it.

Mix-And-Match (geodyncomp)

dg322872 wants to use the cpu of sopale-0-1, but discovers he has no quota there. However he has quota on sopale-0-3 in /state/partition2. What to do?

# ssh to sopale-0-1
ssh -X sopale-0-1
# [optional: start filerunner]
/share/apps/fr &
# make a place to mount the disc of sopale-0-3
mkdir -p geodyncomp-nodes/sopale-0-3
# mount the disc of sopale-0-3
sshfs sopale-0-3:/ geodyncomp-nodes/sopale-0-3
# cd to the directory he wants to use
cd geodyncomp-nodes/sopale-0-3
cd state/
cd partition2
# The first time, he will have to create this directory
# mkdir dg322872
# The directory already exists, so he just cd's
cd dg322872
mkdir -p testing/sopale_nested
cd testing/sopale_nested
# get a copy of the sopale_nested input file
cp -ip /home/dg322872/testing/sopale_nested/sopale_nested_i.Guilmette-makeAnOcean-P1_DIV-20120608_2.0 .
# run sopale_nested (line broken for better formatting)
  start_sopale_nested-2012-06-08_2.1-ompimod-intel-lib-11.0.074-std_large \
  2 no-email nosync sopale_nested_i.Guilmette-makeAnOcean-P1_DIV-20120608_2.0 

Mix-And-Match (scrXm and stable to geodyncomp)

It is possible to use the cpu of scrym, and the disc space on a geodyncomp node. The geodyncomp nodes cannot be seen from the outside world, so a two hop strategy is required. Make the sshfs mounts from the scrXm (or stable) machine to geodyncomp to the geodyncomp node.

# login to scrym
mkdir -p scrXm/geodyncomp
sshfs geodyncomp.ocean.dal.ca:/ scrXm/geodyncomp
ssh -X geodyncomp.ocean.dal.ca
mkdir -p geodyncomp-nodes/sopale-0-0
sshfs sopale-0-0:/ geodyncomp-nodes/sopale-0-0
# create another terminal on scrym
cd scrXm/geodyncomp/geodyncomp-nodes/sopale-0-0/<some directory>
run sopale_nested

This page was last modified on Thursday, 30-May-2013 14:15:21 ADT
Comments to geodynam at dal dot ca