Log In | Users | Register
COSMO Web

Build, Compile and run ICON prior to icon-2.6.0.0

This page gives a Recipe for running ICON releases from C2SM-ICON GitHub organization until icon-2.5.0.3.

If you want to run icon-2.6.0.0 or later, please go to Build, Compile and run ICON icon-2.6.0.0 and later wiki page.

History of this page

October 2017: Here is a summary of the introduction Will Sawyer gave on 17th of October 2017 to build, compile and run the tests of icon-release-2.3.00 on Piz Daint (CSCS).

July 2019 : Updated by C. Siegenthaler for C2SM-ICON organization, on Piz Daint, using CPUs only. Only the compilers gcc and cray are working.

August-October 2019: Adaptation of the recipe for other supercomputers, eg euler.ethz.ch.

January 2020 : Updating part for ICON-LAM

April 2020 : Updating the tag for running on Euler.

Access to the code

To have access to the C2SM-ICON GitHub organization, please contact colombe.siegenthaler@env.ethz.ch.

Configure and compile ICON

This section gives a recipe for configuring and compiling ICON.

0) Set up the right environment

On daint.cscs.ch
Use the modules in /project/d56/modules . To do, either type :
module use /project/d56/modules
or add the following line in your ~/.bash_profile file: MODULEPATH=/project/d56/modules:$MODULEPATH

On euler.ethz.ch
Use the New SPACK software stack on Euler using the LMOD module system (https://scicomp.ethz.ch/wiki/New_SPACK_software_stack_on_Euler):
env2lmod
It allows to use a lot of applications and libraries which are already compiled. Heer is a list of available libraries: https://scicomp.ethz.ch/wiki/Euler_applications_and_libraries

1) Go in a suitable directory for compiling

On daint.cscs.ch
To avoid inconsistencey problem between login nodes and compute nodes, it is recommended to go on $SCRATCH.
cd $SCRATCH
On euler.ethz.ch
Go in a directory which has a lot of space, for example work/climate
cd /cluster/work/climate/[your_username]

2) Clone C2SM ICON repository and go in your new local repository :

git clone git@github.com:C2SM-ICON/icon.git [foldername_local_repo_you_choose] 
cd [foldername_local_repo_you_choose]

If you want to use another branch that the master (for example use ICON-2.4.0 instead of ICON-2.5.0), checkout the branch or tag you want:

git branch -a     # list the available branches
or 
git tag -l        # list the available tags

git checkout branch_you_choose

On euler.ethz.ch
Checkout the tag running on euler (icon-2.5.0.3)
git checkout icon-2.5.0.3

3) Configure using the compiler you want (below [compiler_you_want])

 ./configure --with-fortran=[compiler_you_want]

The list of currently available compilers depending on the machine you are, can be found on the Available Compiler Version for ICON related codes page.

4) Build (compile) the code:

 ./build_command

Run the AMIP case (Global, echam physics)

5) Modify the machine-independent running script located in the run directory (for example run/exp.atm_amip, or create a new one by copiing run/exp.atm_amip into run/exp.my_exp). The latter script is not the running script, the running script "run/exp.my_exp.run" will be created by the command make_runscripts.

 cd run
 edit and adapt exp.atm_amip

For example : You can change the output frequency of the files to 1 month : checkpoint_interval="P1M", restart_interval="P1M", file_interval="P1M". For 6h output data in the file: output_interval="PT6H" The time format is following the ISO-8601 duration string standard, for more info about the time stamp format, please consult the document Namelist_overview.pdf located in the doc folder of your icon distribution.

6) Construct the runscript for your experiment using the command make_runscripts. The output is the running script run/exp.my_exp.run :

 cd ..
 ./make_runscripts my_exp
Note: The command make_runscripts uses the machine-independent setting file located in the run folder run/exp.my_exp (the one edited in point 5), and write a running script for the specific machine which includes the right paths to input files, or the right call to batch system. The command make_runscripts calls config/create_target_header).

7) Adjust your runscript

 cd run
 edit exp.my_exp.run, in particular, watch the #SBATCH header for the number of nodes (for daint.cscs.ch #SBATCH --nodes=XX ) or account number (for daint.cscs.ch #SBATCH --account=XX ).

8) submit the job

On daint.cscs.ch
sbatch --time=01:30:00 exp.my_exp.run 
On euler.ethz.ch
bsub -W [minutes requested or time requested in format HH:MM] exp.my_exp.run

Run an ICON-LAM case (Bernhard's case, NWP physics)

2) The setting file is in the branch icon-lam-pacific_settings
git checkout icon-lam-pacific_settings
The script exp.ICON_limarea_Bernhard_7d is located in the branch icon-lam-pacific_settings.

5) Ask Bernhard for input files and put them in your $SCRATCH: You will have two directories:

$SCRATCH/sim_13k_2017  
$SCRATCH/ICONgrid

6) Follow the point 6 above, my_exp is here: ICON_limarea_Bernhard_7d

The run is running. However, the "horizontal CFL number" is exceed several time at several location. One should probably play with the time step.

Alternate customization

  • To avoid editing each time you use make_runscripts the running script and having to correct some machine-specific command (for example the account number for the batch system, number of nodes requested, etc), it is possible to correct directly the machine-specific commands before using make_runscripts:

For each machine, there is a unction "set_run_target_[machine_your_are_using]()" in config/create_target_header. There, the machine specific commands are defined, especially sbatch/bsub commands. This function will be called when creating the running scripts by make_runscripts.

For example, to set your account number on daint.cscs.ch for all running scripts you will be creating with make_runscripts, go in set_run_target_daint_cpu() (file config/create_target_header) and add in the sbatch commands definition:

#SBATCH --account=[your_project_number]

Input files

Will Sawyer/Colombe Siegenthaler downloaded a lot of data directly from mistral.dkrz.de.

More grid-files can be downloaded if there is an interest, please contact colombe.siegenthaler@env.ethz.ch if you need input files which are missing.

On daint.cscs.ch
All the data should be readable by anyone, please contact colombe.siegenthaler@env.ethz.ch if you have trouble to read the data.

  • You can find the data on store :
/store/c2sm/c2sme/input_gcm/icon

Some of the folder are links to files located on the scratch :

/users/icontest/pool/data/ICON

  • Cases from the DWD tutorial (in 2017) can be found also on store:
/store/c2sm/c2sme/input_gcm/icon/DWD_Tutorial2017

On euler.ethz.ch
  • A preliminary version of the input files can be found here:
/cluster/work/climate/colombsi/icon_input

  • Cases from the DWD tutorial (in 2017):
/cluster/work/climate/colombsi/icon_input/DWD_Tutorial2017

Varia:

* The Cray computer used by DWD is similar to Piz Daint (but without GPU)

* At the demo in october 2017, it was the fisrt time that Will did not need to cross compile. In case a cross-compilation is needed, the configuration command should be the following (please check):

./configure --with-fortran=cray --build=x86_64-cray-linux  --host=x86_64-unknown-linux-gnu

* For access to buildbot (buildbot.zmaw.de/icon/waterfall), you should ask Luis or Reinhard. spacer

This site is managed by the Center for Climate Systems Modeling (C2SM).
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems? Send feedback!