Info |
---|
This page is outdated |
This document explains how to use the implementation of hazard_search.py. Note that it currently works for CyberShake17p9 (v17p9) version only.
ssh your_login@dev01-quakecore.canterbury.ac.nz
(if you don't have a login contact Melody or Daniel)cd /var/www/seisfinder2/CLI
- You will find lots of scripts, enter values in hazard_search_config.ini. A sample config for v17p9 is shown below:
[v17p9]
TMPLOCATION_TO_CREATE_HAZARD_FILES = /rcc/home/projects/quakecore/cybershake/v17p9/Hazard/
EMPIRICAL_FILES_LOCATION = /rcc/home/projects/quakecore/cybershake/v17p9/Empiricals
FAULT_LIST_CSV_FILE_LOCATION = /rcc/home/projects/quakecore/cybershake/v17p9/cyber_shake_file_v17p9.csv
- The one that you need to run is
hazard_search.py
.python hazard_search.py -h
will show the help for the script.python hazard_search.py -l
returns all the simulation groups we currently have (disabled and we only have v17p9).python hazard_search.py -s cybershake_version latitude longitude IM
(or--single cybershake_version latitude longitude IM
) will create a temporary directory for you and will copy (if they exist) the empirical file for the closest station that we have for the given latitude and longitude. It will print some instructions on how to calculate hazard and deaggregation.python hazard_search.py -m cybershake_version csv_file IM
(or--multi cybershake_version csv_file IM
) takes as input a CSV file containing latitude,longitude lines for several locations of interest. As previously, it will copy all the relevant empirical files to a temporary folder. The instructions printed will be shorter, but you will be provided some bash scripts to automate the execution.
Running hazard search for entire Cybershake on Mahuika
...
Code Block |
---|
pip install geopy, psycopg2, numpy, pyarrow shapely --user |
Consider upgrading pip by
Code Block |
---|
pip upgrading pip --user |
Create a temporary directory. Let's say ~/tmp and copy 2 files to here
...
then update PATH to use the pip in $HOME/.local/bin/pip
Setup
Your Cybershake root directory must contain Runs and Data.
- Runs: Make sure all the IM Agg have been computed and stored under each fault's Runs directory. ie. /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6/Runs/XXXX/IM_agg
- Data: Containing VMs and Sources
Go to the Cybershake root directory (eg. /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6) and execute the setup.py
Code Block | ||
---|---|---|
| ||
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6$ python /nesi/project/nesi00213/deploy/seisfinder2/ |
...
tool/setup.py --help
usage: setup.py [-h] [-n NAME] cybershake_dir empirical_dir ll_file
positional arguments:
cybershake_dir Cybershake Root directory eg.
/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6
empirical_dir Empirical data root directory eg.
/nesi/nobackup/nesi00213/empiricals/v18p5
ll_file Path to the .ll file
optional arguments:
-h, --help show this help message and exit
-n NAME, --name NAME Optional name for Cybershake. |
module add Python/2.7.14-gimkl-2017a
The following example was taken from v18p6_3 directory.
Code Block |
---|
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3$ python /nesi/project/nesi00213/deploy/seisfinder2/tool/setup.py . /nesi/nobackup/nesi00213/empiricals/v18p6 non_uniform_whole_nz_with_real_stations-hh400_v18p6.ll
/scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/imdb.h5
{empirical_dir: /nesi/nobackup/nesi00213/empiricals/v18p6, fault_list: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/fault_list.csv,
hazard_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard,
ll: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3 |
...
Code Block |
---|
[database] SQLITE_DB_PATH = /home/baes/seisfinderdb.db [v18p6] TMPLOCATION_TO_CREATE_HAZARD_FILES = /home/baes/cybershake/v18p6/Hazard/ EMPIRICAL_FILES_LOCATION = /nesi/project/nesi00213/deploy/empiricals/v18p5 FAULT_LIST_CSV_FILE_LOCATION = /nesi/project/nesi00213/deploy/cybershake/v18p6/fault_list.csv LL_FILE = /nesi/project/nesi00213/deploy/cybershake/v18p6/non_uniform_whole_nz_with_real_stations-hh400_v18p6_land.ll IM_PATH= /nesi/project, name: v18p6_3, root_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/deployRunFolder/cybershakeCybershake/v18p6/IMs |
3. Edit execute_hazard_search.sl, especially the line starting with "srun" at the bottom.
Code Block |
---|
#!/bin/bash # script version: slurm # Please modify this file as needed, this is just a sample #SBATCH --job-name=hazard_search_multi #SBATCH --account=nesi00213 #SBATCH --partition=prepost #SBATCH --ntasks=1 ####SBATCH --cpus-per-task=36 #SBATCH --time=00:59:00 #SBATCH --output hazard_search_multi-%j.out #SBATCH --error hazard_search_multi-%j.err ###SBATCH --mail-type=all ###SBATCH --mail-user=test@test.com ###SBATCH --mem-per-cpu=16G ###SBATCH -C avx #OpenMP+Hyperthreading works well for VM ###SBATCH --hint=nomultithread ## END HEADER source machine_env.sh date srun python /nesi/project/nesi00213/deploy/seisfinder2/CLI/hazard_search.py -m v18p6 /nesi/project/nesi00213/deploy/cybershake/v18p6_3, runs_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Runs, srf_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Data/Sources} Config file has been successfully written: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/config.yaml Warning: More faults found than simulated results : Ignoring these ['OtaraWest02', 'OtaraWest01', 'Wairaka05', 'OtaraEast03', 'FiordSZ09', 'FiordSZ03', 'TaurTrE02', 'Tuakana11', 'Ohena02', 'MaungatiE01', 'Ohena01'] Output CSV successfully written: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/fault_list.csv /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/non_uniform_whole_nz_with_real_stations-hh400_v18p6_land_lat_long.csv PGA date |
To generate a list of locations .csv from a .ll file, you can use the following command
Code Block |
---|
cat XXX.ll |awk '{ print $2", "$1}' > XXX.csv |
4. Submit this job.
Code Block |
---|
sbatch execute_hazard_search.sl |
5. When the job has completed, check the last -10 lines of .out log file.
Code Block |
---|
baes@mahuika02: ~/tmp$ tail -10 hazard_search_multi-85422.out
Instructions in the multicase are not written in extense as it is not useful. Please use the scripts below
Please go to hazard/
Then run the scripts created as:
bash /home/baes/cybershake/v18p6/Hazard/hazard_search_5F73LY_baes/hazard_calcs.sh
bash /home/baes/cybershake/v18p6/Hazard/hazard_search_5F73LY_baes/deagg_calcs_im_value.sh
bash /home/baes/cybershake/v18p6/Hazard/hazard_search_5F73LY_baes/deagg_calcs_exceedance_rate.sh
Note: The IM_VALUE and EXCEEDANCE should be adjusted on the deagg_* scripts
Mon Sep 3 03:30:39 UTC 2018 |
This means it has created a temp directory
/home/baes/cybershake/v18p6/Hazard/hazard_search_5F73LY_baes
and placed empirical files (symbolic links) and bash scripts that contain Python commands.
6. Let's call this temp location a working directory.
7. There is hazard_calcs.sh
in the working directory. For entire cybershake, this can be very very long. We will be using SLURM's parallel for loop to run this in parallel, but it needs to be split to avoid SLURM complaining about too many lines. This involves quite a bit of steps, but it can be largely automated by executing prepare_hpc_hazard_calc.sh
. Go to the working directory, and execute it. You just need to supply the path to the working directory.
.ll
Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/imdb_calc.sl
sbatch imdb_calc.sl
Submitted batch job 1874061 |
This creates a config.yaml that contains the all the necessary config info, creates fault lists, converts .ll into .csv etc, and builds IMDB by crafting a .sl script and automatically submitting it. (This takes about an hour)
Info |
---|
In earlier versions, setup.py used to produce hazard_calc_X.sl scripts. We have a better solution, and no longer support this. This step will be deprecated. Written /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_calc_SA_0p5.sl |
A sample config.yaml looks like this
Code Block |
---|
empirical_dir: /nesi/nobackup/nesi00213/empiricals/v18p6
fault_list: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/fault_list.csv
hazard_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard
ll: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/non_uniform_whole_nz_with_real_stations-hh400_v18p6_land.ll
name: v18p6_3
root_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3
runs_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Runs
srf_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Data/Sources
|
Run
Go to Cybershake root directory. (Make sure machine_env.sh has been imported and)
Code Block |
---|
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --cs-version v18p6_3 --im SA_5p0 --location -42.7012 172.8003 --hazard
|
This calculates hazard for the given location for SA_5p0.
You can use a CSV file that contains latitude and longitude of multiple locations.
Code Block |
---|
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --cs-version v18p6_3 --im SA_5p0 --csv-location /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/test.csv --hazard
|
Note that once you set parameters, cs-version, im and locations etc., the system remembers them. So you can simply execute the following command and get the same result
Code Block |
---|
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --hazard
|
This can be useful if you want to compute deagg for the same location and same setting. You just need to put additional parameters such as exceedance as below.
Code Block |
---|
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --deagg --exceedance 0.0051
|
You can make the system forget the setting by --clear switch
Code Block |
---|
baes@mahuika02: ~$ python |
Code Block |
baes@mahuika02: ~/cybershake/v18p6/Hazard/hazard_search_5F73LY_baes $ bash /nesi/project/nesi00213/deploy/seisfinder2/CLI/hazard/mahuika/prepare_hpc_hazard_calc.sh . |
The above does EVERYTHING for you. It copies the .sl template to the working directory, splits the hazard_calc.sh, and generates all the .sl files ready for submission.
site/site_search.py --clear
|
If you put --script option, it won't do the calculation. Instead it will output a file hazard_calcs.sh
in the current directory. This is useful if you wish to use NeSI to run hazard calculation for all locations and produce hazard map.
Code Block |
---|
baes@mahuika02: ~$ python /nesi/project/nesi00213/deploy/seisfinder2/site/site_search.py --cs-version v18p6_3 --im SA_5p0 --csv-location /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_2/test.csv --hazard --script
|
Suppose there is ~/hazard_calcs.sh. You can run prepare_hpc_hazard.calcs.sh for SLURM submission.
Code Block |
---|
baes@mahuika02: ~$ bash /nesi/project/nesi00213/deploy/seisfinder2/site/mahuika/prepare_hpc_hazard_calc.sh
Found working directory /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/SA_5p0
Found 80 unique locations
Making a slurm script for processing locations 0 - 80
....Done!!
Go to the following directory and run submit_all.sh
/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/SA_5p0/slurm
|
Everything you need is placed in the directory. Go to the directory and execute submit_all.sh (SLURM doesn't like to process more than 10000 lines. If there are more locations than the limit, it will create multiple .sl scripts and submit_all.sh will submit them all for you)
Code Block |
---|
baes@mahuika02: /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/SA_5p0/slurm$ ./submit_all.sh |
Code Block |
---|
baes@mahuika02: ~/cybershake/v18p6/Hazard/hazard_search_5F73LY_baes$ bash ./submit_all.sh Submitted batch job 69390 Submitted batch job 69391 Submitted batch job 69392 Submitted batch job 69393 Submitted batch job 69394 Submitted batch job 69395 Submitted batch job 69396 Submitted batch job 69397 Submitted batch job 69398 Submitted batch job 69399 Submitted batch job 69400 Submitted batch job 69401 Submitted batch job 69402 Submitted batch job 69403 Submitted batch job 6940469390 Submitted batch job 6940569391 Submitted batch job 6940669392 Submitted batch job 69407 Submitted batch job 6940869393 ... |
9. Hazard map: When everything is completed, you can submit the following .sl script that has been automatically generated for you. Just edit the exceedance value if required. For details, see https://github.com/ucgmsim/seisfinder2/tree/master/CLI/hazard
Code Block |
---|
baes@mahuika02: ~/cybershake/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6_3/Hazard/hazard_search_5F73LY_baes$SA_5p0/slurm$ sbatch execute_hazard_map.sl |
Sample Outputs
Hazard Curve
site_search.py --cs-version 18.6 --location -43.89 170.435 --hazard --im PGA
Deaggregation
./site_search.py --cs-version 18.6 --location -43.89 170.435 --deagg --im PGA --exceedance 0.01
View file | ||||
---|---|---|---|---|
|
View file | ||||
---|---|---|---|---|
|
GM Selection
site_search.py --cs-version 18.6 --location -43.89 170.435 --gmsel --im PGA --imvalue 0.1
View file | ||||
---|---|---|---|---|
|