Running hazard search for entire Cybershake on Mahuika
You may wish to install python packages geopy, psycopg2, numpy, pyarrow, etc. Use pip install --user to install missing packages.
pip install geopy psycopg2 shapely --user
Consider upgrading pip by
pip upgrading pip --user
then update PATH to use the pip in $HOME/.local/bin/pip
Setup
Your Cybershake root directory must contain Runs and Data.
- Runs: Make sure all the IM Agg have been computed and stored under each fault's Runs directory. ie. /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6/Runs/XXXX/IM_agg
- Data: Containing VMs and Sources
Go to the Cybershake root directory (eg. /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6) and execute the setup.py
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6$ python /nesi/project/nesi00213/deploy/seisfinder2/tool/setup.py --help
usage: setup.py [-h] [-n NAME] cybershake_dir empirical_dir ll_file
positional arguments:
cybershake_dir Cybershake Root directory eg.
/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6
empirical_dir Empirical data root directory eg.
/nesi/nobackup/nesi00213/empiricals/v18p5
ll_file Path to the .ll file
optional arguments:
-h, --help show this help message and exit
-n NAME, --name NAME Optional name for Cybershake.
module add Python/2.7.14-gimkl-2017a
The following example was taken from v18p6_3 directory.
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3$ python /nesi/project/nesi00213/deploy/seisfinder2/tool/setup.py . /nesi/nobackup/nesi00213/empiricals/v18p6 non_uniform_whole_nz_with_real_stations-hh400_v18p6.ll
/scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/imdb.h5
{empirical_dir: /nesi/nobackup/nesi00213/empiricals/v18p6, fault_list: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/fault_list.csv,
hazard_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard,
ll: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/non_uniform_whole_nz_with_real_stations-hh400_v18p6_land.ll,
name: v18p6_3, root_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3,
runs_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Runs,
srf_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Data/Sources}
Config file has been successfully written: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/config.yaml
Warning: More faults found than simulated results : Ignoring these
['OtaraWest02', 'OtaraWest01', 'Wairaka05', 'OtaraEast03', 'FiordSZ09', 'FiordSZ03', 'TaurTrE02', 'Tuakana11', 'Ohena02', 'MaungatiE01', 'Ohena01']
Output CSV successfully written: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/fault_list.csv
/scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/non_uniform_whole_nz_with_real_stations-hh400_v18p6_land.ll
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/imdb_calc.sl
sbatch imdb_calc.sl
Submitted batch job 1874061
This creates a config.yaml that contains the all the necessary config info, creates fault lists, converts .ll into .csv etc, and builds IMDB by crafting a .sl script and automatically submitting it. (This takes about an hour)
In earlier versions, setup.py used to produce hazard_calc_X.sl scripts. We have a better solution, and no longer support this. This step will be deprecated.
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_0p5.sl
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_PGA.sl
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_0p2.tar.sl
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_0p1.sl
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_5p0.tar.sl
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_0p2.sl
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_3p0.sl
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_2p0.sl
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_5p0.sl
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_1p0.sl
A sample config.yaml looks like this
empirical_dir: /nesi/nobackup/nesi00213/empiricals/v18p6 fault_list: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/fault_list.csv hazard_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard ll: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/non_uniform_whole_nz_with_real_stations-hh400_v18p6_land.ll name: v18p6_3 root_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3 runs_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Runs srf_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Data/Sources
Run
Go to Cybershake root directory. (Make sure machine_env.sh has been imported and)
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --cs-version v18p6_3 --im SA_5p0 --location -42.7012 172.8003 --hazard
This calculates hazard for the given location for SA_5p0.
You can use a CSV file that contains latitude and longitude of multiple locations.
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --cs-version v18p6_3 --im SA_5p0 --csv-location /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/test.csv --hazard
Note that once you set parameters, cs-version, im and locations etc., the system remembers them. So you can simply execute the following command and get the same result
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --hazard
This can be useful if you want to compute deagg for the same location and same setting. You just need to put additional parameters such as exceedance as below.
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --deagg --exceedance 0.0051
You can make the system forget the setting by --clear switch
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --clear
If you put --script option, it won't do the calculation. Instead it will output a file hazard_calcs.sh in the current directory. This is useful if you wish to use NeSI to run hazard calculation for all locations and produce hazard map.
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --cs-version v18p6_3 --im SA_5p0 --csv-location /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/test.csv --hazard --script
Suppose there is ~/hazard_calcs.sh. You can run prepare_hpc_hazard.calcs.sh for SLURM submission.
baes@mahuika02: ~$ bash /nesi/project/nesi00213/deploy/seisfinder2/site/mahuika/prepare_hpc_hazard_calc.sh Found working directory /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/SA_5p0 Found 80 unique locations Making a slurm script for processing locations 0 - 80 ....Done!! Go to the following directory and run submit_all.sh /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/SA_5p0/slurm
Everything you need is placed in the directory. Go to the directory and execute submit_all.sh (SLURM doesn't like to process more than 10000 lines. If there are more locations than the limit, it will create multiple .sl scripts and submit_all.sh will submit them all for you)
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/SA_5p0/slurm$ ./submit_all.sh Submitted batch job 69390 Submitted batch job 69391 Submitted batch job 69392 Submitted batch job 69393 ...
Hazard map: When everything is completed, you can submit the following .sl script that has been automatically generated for you. Just edit the exceedance value if required. For details, see https://github.com/ucgmsim/seisfinder2/tree/master/CLI/hazard
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/SA_5p0/slurm$ sbatch execute_hazard_map.sl
Sample Outputs
Hazard Curve
site_search.py --cs-version 18.6 --location -43.89 170.435 --hazard --im PGA
Deaggregation
./site_search.py --cs-version 18.6 --location -43.89 170.435 --deagg --im PGA --exceedance 0.01
GM Selection
site_search.py --cs-version 18.6 --location -43.89 170.435 --gmsel --im PGA --imvalue 0.1
