...
- Only 1 slurm script to submit.
- Automated VM integrity test.
Code Block | ||
---|---|---|
| ||
#!/bin/bash # script version: slurm # Please modify this file as needed, this is just a sample #SBATCH --job-name=nhm2srf_vm #SBATCH --account=nesi00213 #SBATCH --partition=NeSI #SBATCH --ntasks=1 #SBATCH --time=03:00:00 #SBATCH --output nhm2srf_vm-%j.out #SBATCH --error nhm2srf_vm-%j.err ###SBATCH --mail-type=all ###SBATCH --mail-user=test@test.com ###SBATCH --mem-per-cpu=16G ###SBATCH -C avx #SBATCH --hint=nomultithread ## END HEADER source machine_env.sh # user specific parameters export NUMPROCS=40 export SELECTION_FILE=`pwd`/nhm_selection_geo_si.txt #nhm_selection_sample2.txt # # important paths export SRFGENPATH=/nesi/nobackup/nesi00213/tmp/Pre-processing/SrfGen export QCOREPATH=/nesi/nobackup/nesi00213/tmp/qcore export GMPEMODELSPATH=/nesi/nobackup/nesi00213/tmp/post-processing/im_processing/computations/GMPE_models # leave these alone export PYTHONPATH=$QCOREPATH:$PYTHONPATH:$SRFGENPATH:$GMPEMODELSPATH export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/nesi/nobackup/nesi00213/gmt/gmt-stable/lib64 # to fix HDF5 file locking issue https://github.com/ALPSCore/ALPSCore/issues/348 export HDF5_USE_FILE_LOCKING=FALSE echo "SRF generation starts" date python2 $SRFGENPATH/NHM/nhm2srf.py $SELECTION_FILE -n $NUMPROCS echo "SRF generation completed" date echo "VM generation starts" srun python2 $SRFGENPATH/srfinfo2vm.py "autosrf/*/Srf/*.info" -n $NUMPROCS date echo "VM generation completed" |
When submitted and executed, it creates two directories `autosrf` and `autovm`
Code Block |
---|
baes@kupe01: /scale_akl_nobackup/filesets/nobackup/nesi00213/tmp/auto_preproc/nhm_test/autosrf$ ls
AhuririR Caswell4 FiordMar1&2 Hundalee MilB5GeoR2 NorthCant13 Springbank
Akatore Caswell5 FiordSZ03 Hunter MilfordB1 NorthCant2 Springfield
Albury CaswellH10 FiordSZ09 HuttPeelNorth MoonlightNth NorthCant4 Spylaw
AlpineF2K CaswellH67 FiveFingers HuttPeelSouth MoonlightSth NorthCant8 Stonewall
AlpineK2T CaswellH8 Fowlers Hyde MS01 OhariuS Swedge1
AlpineR CaswellH9 FoxPk Inangahua MS02 OldMan Swedge2
Ashley CBalleny GeorgeR1 IrishmanC MS04 Omihi Swedge3
AwatereNE Chalky1to3 Gimmerburn JorKekCha MS05 Opawa Swedge5
AwatereSW Chalky4to8 Grandview JorKekNeed MS09 Ostler Swedge6to10
AwatNEVer Cheeseman Hanmer KaiwaraN MtWhite Otematata TaieriR
AwatNEVerCl ClarenceCentr Harper KaiwaraS Nevis PaparoaRF Takitimu1
Barefell ClarenceNE Hauroko Kakapo NMFZ1819 Pegasus1nw Takitimu2
Billys ClarenceSW HikWgtnmax KekerenguBF NMFZ4647 Pisa TeAnau
BlueLk Cust HikWgtnmin KekNeed NMFZB0 Port2GreyL Torlesse
BlueMtn Cw4Swedge411 Hokonui Kelly NMFZB1 Port2GreyS UpperSlope
BooBooALL Cwedge123 Hollyford Kirkliston NMFZB2 Poulter WaimeaN
Brothers Dalgety Hope1888 LeesV NMFZE1 QuartzCk WaimeaS
Browning DoubleHill HopeConway LindisP NMFZE2 Raggedy Waipiata
BrunAnt Dryburgh HopeConwayOS LkHeron NMFZF1 Rakaia WairarapNich
CardronaN Dunstan HopeCW logfile.txt NMFZF2 Ranfurly Wairau
CardronaS Esk HopeTARA LongVlly NorthBranch Settlement Waitangi
Caswell1 fault_traces.gmt HopeTeRapa1n2 Lowry NorthCant1 SFiordMg13 Waitohi
Caswell211 FernGully Hororata Lyell NorthCant10 SFiordMg1to9 WharaToCampB
Caswell3 Fidget HumpR Maimai NorthCant11 Solander WhiteCk
|
Code Block |
---|
baes@kupe01: /scale_akl_nobackup/filesets/nobackup/nesi00213/tmp/auto_preproc/nhm_test/autosrf/Akatore$ ls
Srf Stoch
baes@kupe01: /scale_akl_nobackup/filesets/nobackup/nesi00213/tmp/auto_preproc/nhm_test/autosrf/Akatore$ cd Srf
baes@kupe01: /scale_akl_nobackup/filesets/nobackup/nesi00213/tmp/auto_preproc/nhm_test/autosrf/Akatore/Srf$ ls
Akatore_HYP01-04_S1244.info Akatore_HYP02-04_S1284.info Akatore_HYP03-04_S1324.info
Akatore_HYP01-04_S1244.srf Akatore_HYP02-04_S1284.srf Akatore_HYP03-04_S1324.srf
Akatore_HYP01-04_S1254.info Akatore_HYP02-04_S1294.info Akatore_HYP04-04_S1334.info
Akatore_HYP01-04_S1254.srf Akatore_HYP02-04_S1294.srf Akatore_HYP04-04_S1334.srf
Akatore_HYP01-04_S1264.info Akatore_HYP03-04_S1304.info Akatore_HYP04-04_S1344.info
Akatore_HYP01-04_S1264.srf Akatore_HYP03-04_S1304.srf Akatore_HYP04-04_S1344.srf
Akatore_HYP02-04_S1274.info Akatore_HYP03-04_S1314.info Akatore_HYP04-04_S1354.info
Akatore_HYP02-04_S1274.srf Akatore_HYP03-04_S1314.srf Akatore_HYP04-04_S1354.srf
baes@kupe01: /scale_akl_nobackup/filesets/nobackup/nesi00213/tmp/auto_preproc/nhm_test/autosrf/Akatore/Srf$
|
Similarly, `autovm` directory contains velocity model files for each location.
Code Block |
---|
baes@kupe01: /scale_akl_nobackup/filesets/nobackup/nesi00213/tmp/auto_preproc/nhm_test/autovm/Akatore$ ls
gridfile_rt01-h0.400 model_coords_rt01-h0.400 params_vel.py VeloModCorners.txt
gridout_rt01-h0.400 model_params_rt01-h0.400 params_vel.pyc vp3dfile.p
model_bounds_rt01-h0.400 nzvm.cfg rho3dfile.d vs3dfile.s
|
To Do
- Technical debts: /nesi/nobackup/qcore /nesi/nobackup/post-processing need to be updated, and other codes that depend on old versions need to be revised
...