...
This creates a config.yaml that contains the all the necessary config info, then creates CSVs out of IM Agg, and add the data into the DB, creates fault lists, converts .ll into .csv etc.
A sample config.yaml looks like this
| Code Block |
|---|
csv_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/CSV db: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/v18p6.db empirical_dir: /nesi/nobackup/nesi00213/empiricals/v18p5 fault_list: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/fault_list.csv hazard_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/hazardHazard ims_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/IMs ll: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/non_uniform_whole_nz_with_real_stations-hh400_v18p6.ll name: v18p6 root_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6 runs_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/Runs srf_dir: /scale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/Data/Sources |
Edit `common/constants.py` [!!! Should be in the seisfinder root directory as a config file. This needs an overhaul ]. You need to specify DB_PATH and TEMP_DIR
| Code Block |
|---|
import os
DB_PATH = '/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6/v18p6.db'
VERSION_OPTIONS = ["18p9p3"]
SEARCH_OPTIONS = ["event", "site"]
EVENT_SEARCH = 1
SITE_SEARCH = 2
CLI_PROGRAMS_OPTIONS = ["guided", "argument"]
TIME_OPTIONS = ["historical", "future"]
LOCATION_OPTIONS = ["single", "multiple"]
SINGLE_LOCATION = 1
MULTIPLE_LOCATION = 2
TEMP_DIR = "/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6/hazard/" # location where any user can read, write and execute
TMPLOCATION_TO_CREATE_HAZARD_FILES = os.path.join(TEMP_DIR,"Hazard")
SEISFINDER_RUN_CMD = 'python seisfinder.py' |
Notice VERSION_OPTIONS above. Edit /common/versions/18p9p3.ini. Specifically, we will be enabling site search for v18p6. [!!!! Needs an overhaul !!!!]
| Code Block |
|---|
... [site] quakes = v18p6 ... |
Run
The setup process has created Hazard under Cybershake directory. eg. /nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6/Hazard
Navigate to the directory and find .sl scripts that have been already created for you.
A sample hazard_calc_SA_5p0.slEdit execute_hazard_search.sl, especially the line starting with "srun" at the bottom.
| Code Block |
|---|
#!/bin/bash # script version: slurm # Please modify this file as needed, this is just a sample #SBATCH --job-name=hazard_search_multi #SBATCH --account=nesi00213 #SBATCH --partition=prepost #SBATCH --ntasks=1 ####SBATCH --cpus-per-task=36 #SBATCH --time=00:59:00 #SBATCH --output hazard_search_multi-%j.out #SBATCH --error hazard_search_multi-%j.err ###SBATCH --mail-type=all ###SBATCH --mail-user=test@test.com ###SBATCH --mem-per-cpu=16G ###SBATCH -C avx #OpenMP+Hyperthreading works well for VM ###SBATCH --hint=nomultithread ## END HEADER SITECODE=/nesi/project/nesi00213/deploy/seisfinder2/site source $SITECODE/mahuika/machine_env.sh date srun python /nesi/project$SITECODE/site_search.py -d /scale_wlg_nobackup/filesets/nobackup/nesi00213/deployRunFolder/seisfinder2/site/site_search.pyCybershake/v18p6 -m v18p6 /nesiscale_wlg_nobackup/filesets/nobackup/nesi00213/RunFolder/Cybershake/v18p6/non_uniform_whole_nz_with_real_stations-hh400_v18p6_land.csv SA_5p0 date |
Submit this job.
| Code Block |
|---|
sbatch executehazard_calc_hazardSA_search5p0.sl |
When the job has completed, check the last -10 lines of .out log file.
...
| Code Block |
|---|
baes@mahuika02: ~/tmp$/nesi/nobackup/nesi00213/RunFolder/Cybershake/v18p6/Hazard$$ tail -10 hazard_search_multi-85422.out Instructions in the multicase are not written in extense as it is not useful. Please use the scripts below Please go to hazard/ Then run the scripts created as: bash /home/baes/cybershake/v18p6/Hazard/hazard_search_v18p6_SA_5p0/hazard_calcs.sh bash /home/baes/cybershake/v18p6/Hazard/hazard_search_v18p6_SA_5p0/deagg_calcs_im_value.sh bash /home/baes/cybershake/v18p6/Hazard/hazard_search_v18p6_SA_5p0/deagg_calcs_exceedance_rate.sh Note: The IM_VALUE and EXCEEDANCE should be adjusted on the deagg_* scripts Mon Sep 3 03:30:39 UTC 2018 |
...
Let's call this temp location a working directory.
There is hazard_calcs.sh in the working directory. For entire cybershake, this can be very very long. We will be using SLURM's parallel for loop to run this in parallel, but it needs to be split to avoid SLURM complaining about too many lines. This involves quite a bit of steps, but it can be largely automated by executing prepare_hpc_hazard_calc.sh. Go to the working directory, and execute it. You just need to supply the path to the working directory.
...