...
Make sure all the IM Agg have been computed and stored under each fault of Cybershake IMs directory. ie. /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6/IMs/XXXX/IM_agg
Setup
Setup part may need updating. imdb_create needs to be added too
| Code Block |
|---|
module add Python/2.7.14-gimkl-2017a
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6$ python /nesi/project/nesi00213/deploy/seisfinder2/tool/setup.py --help
usage: setup.py [-h] [-n NAME] cybershake_dir empirical_dir ll_file
positional arguments:
cybershake_dir Cybershake Root directory eg.
/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6
empirical_dir Empirical data root directory eg.
/nesi/nobackup/nesi00213/empiricals/v18p5
ll_file Path to the .ll file
optional arguments:
-h, --help show this help message and exit
-n NAME, --name NAME Optional name for Cybershake |
...
A sample config.yaml looks like this
| Code Block |
|---|
csv_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/CSV db: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/v18p6.db empirical_dir: /nesi/nobackup/nesi00213/empiricals/v18p5v18p6 fault_list: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/fault_list.csv hazard_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/Hazard ims_dirll: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/IMs ll: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/non_uniform_whole_nz_with_real_stations-hh400_v18p6_land.ll name: v18p6_2 root_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2 runs_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/Runs srf_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/Data/Sources |
Notice VERSION_OPTIONS above. Edit /common/versions/18p9p3.ini. Specifically, we will be enabling site search for v18p6. [!!!! Needs an overhaul !!!!]
| Code Block |
|---|
...
[site]
quakes = v18p6
... |
Run
The setup process has created Hazard under Cybershake directory. eg. /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6/Hazard
Navigate to the directory and find .sl scripts that have been already created for you.
A sample hazard_calc_SA_5p0.sl
Run
| Code Block |
|---|
baes@mahuika02: ~$ python ~/seisfinder2/site/seisfinder.py --cs-version v18p6_2 --im SA_5p0 --location -42.7012 172.8003 --hazard
|
This calculates hazard for the given location for SA_5p0.
You can use a CSV file that contains latitude and longitude of multiple locations.
| Code Block |
|---|
baes@mahuika02: ~$ python ~/seisfinder2/site/seisfinder.py --cs-version v18p6_2 --im SA_5p0 --csv /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/test.csv --hazard
|
If you put --script option, it won't do the calculation. Instead it will output a file hazard_calcs.sh in the current directory. This is useful if you wish to use NeSI to run hazard calculation for all locations.
| Code Block |
|---|
baes@mahuika02: ~$ python ~/seisfinder2/site/seisfinder.py --cs-version v18p6_2 --im SA_5p0 --csv /nesi/ |
| Code Block |
#!/bin/bash # script version: slurm # Please modify this file as needed, this is just a sample #SBATCH --job-name=hazard_search_multi #SBATCH --account=nesi00213 #SBATCH --partition=prepost #SBATCH --ntasks=1 ####SBATCH --cpus-per-task=36 #SBATCH --time=00:59:00 #SBATCH --output hazard_search_multi-%j.out #SBATCH --error hazard_search_multi-%j.err ###SBATCH --mail-type=all ###SBATCH --mail-user=test@test.com ###SBATCH --mem-per-cpu=16G ###SBATCH -C avx #OpenMP+Hyperthreading works well for VM ###SBATCH --hint=nomultithread ## END HEADER SITECODE=/nesi/project/nesi00213/deploy/seisfinder2/site source $SITECODE/mahuika/machine_env.sh date srun python $SITECODE/site_search.py -d /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6 -m v18p6 /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/non_uniform_whole_nz_with_real_stations-hh400_v18p6_2/test.csv SA_5p0 date |
Submit this job.
| Code Block |
|---|
sbatch hazard_calc_SA_5p0.sl |
When the job has completed, check the last -10 lines of .out log file.
| Code Block |
|---|
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6/Hazard$$ tail -10 hazard_search_multi-85422.out
Instructions in the multicase are not written in extense as it is not useful. Please use the scripts below
Please go to hazard/
Then run the scripts created as:
bash /home/baes/cybershake/v18p6/Hazard/hazard_search_v18p6_SA_5p0/hazard_calcs.sh
bash /home/baes/cybershake/v18p6/Hazard/hazard_search_v18p6_SA_5p0/deagg_calcs_im_value.sh
bash /home/baes/cybershake/v18p6/Hazard/hazard_search_v18p6_SA_5p0/deagg_calcs_exceedance_rate.sh
Note: The IM_VALUE and EXCEEDANCE should be adjusted on the deagg_* scripts
Mon Sep 3 03:30:39 UTC 2018 |
This means it has created a temp directory /home/baes/cybershake/v18p6/Hazard/hazard_search_v18p6_SA_5p0 and placed empirical files (symbolic links) and bash scripts that contain Python commands.
Let's call this temp location a working directory.
...
--hazard --script
|
Suppose there is ~/hazard_calcs.sh. You can run prepare_hpc_hazard.calcs.sh for SLURM submission.
| Code Block |
|---|
baes@mahuika02: ~$ bash /nesi/project/nesi00213/deploy/seisfinder2/site/mahuika/prepare_hpc_hazard_calc.sh
Found working directory /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/Hazard/SA_5p0
Found 80 unique locations
Making a slurm script for processing locations 0 - 80
....Done!!
Go to the following directory and run submit_all.sh
/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/Hazard/SA_5p0/slurm
|
Everything you need is placed in the directory. Go to the directory and execute submit_all.sh (SLURM doesn't like to process more than 10000 lines. If there are more locations than the limit, it will create multiple .sl scripts and submit_all.sh will submit them all for you)
| Code Block |
|---|
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/Hazard/SA_5p0/slurm$ |
| Code Block |
|---|
baes@mahuika02: ~/cybershake/v18p6/Hazard/hazard_search_v18p6_SA_5p0 $ bash /nesi/project/nesi00213/deploy/seisfinder2/site/mahuika/prepare_hpc_hazard_calc.sh .
|
The above does EVERYTHING for you. It copies the .sl template to the working directory, splits the hazard_calc.sh, and generates all the .sl files ready for submission.
| Code Block |
|---|
baes@mahuika02: ~/cybershake/v18p6/Hazard/hazard_search_v18p6_SA_5p0$ bash ./submit_all.sh Submitted batch job 69390 Submitted batch job 69391 Submitted batch job 69392 Submitted batch job 69393 Submitted batch job 69394 Submitted batch job 69395 Submitted batch job 69396 Submitted batch job 69397 Submitted batch job 69398 Submitted batch job 69399 Submitted batch job 69400 Submitted batch job 69401 Submitted batch job 69402 Submitted batch job 69403 Submitted batch job 69404 Submitted batch job 69405 Submitted batch job 69406 Submitted batch job 69407 Submitted batch job 69408... |
Hazard map: When everything is completed, you can submit the following .sl script that has been automatically generated for you. Just edit the exceedance value if required. For details, see https://github.com/ucgmsim/seisfinder2/tree/master/CLI/hazard
| Code Block |
|---|
baes@mahuika02: ~/cybershake/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/Hazard/hazard_search_v18p6_SA_5p0$5p0/slurm$ sbatch execute_hazard_map.sl |
...