Current readme for real-time ground motion simulation
- Source generation (Viktor/Hoby)
- Srf Generation/Plotting
- Manually extract location, dip,strike, rake from Mw tensor (Ristau) into setSrfParams.py
- Change TYPE=2 (point source to finite fault)
- Run createSRF.py
- Velocity model generation
- Preliminary steps to setup on hypocentre (if required)
- Clone the AutoVMGen repo https://github.com/ucgmsim/Auto-Vel-Mod-Generation.git
- Clone the VM repo into the AutoVMGen directory (or create a symlink) https://github.com/ucgmsim/Velocity-Model.git
- Compile the VM code by executing 'make parallel'
- Run generate_parameters.py
- Uses hypocentre Mw, depth, lat, lon (and others optional inputs e.g. hh)
- creates a file called params_vel.py which has the velocity model parameters (Lx, Ly, Lz, hh, model origin, model rotation) and simulation duration
- Run investigateDomain.py
- Loads the parameters stored in params_vel.py
- Outputs a plot of the VM domain on a map (found in the directory 'Domain')
- A simulation duration estimate is printed to the screen (assuming 512 core simulation)
- If a change in velocity model parameters is required, update desired variables in params_vel.py, run validateparameters.py and then rerun investigateDomain.py (can repeat this step until satisfied with the VM domain)
- Run genDomain.py
- Generates the VM and stores it the directory Rapid_Domain, with extracted slices and domain on map plots.
- Preliminary steps to setup on hypocentre (if required)
- Observed ground motions
- Ahsan
NOTE: Installation is required only once, unless there is a need to update. geoNet
package is self contained and should work on your pcs and laptops too.0) INSTALL geoNet.
From
https://github.com/ucgmsim/Pre-processing/tree/master/geoNet
clone geoNet and follow the installation instructions. Copy event_info.txt found at geoNet/examples and modify. Then do the following:
from geoNet import realtime
realtime.run("event_info.txt")
Alternatively performs the steps (1) to (5) outlined below.1) GET THE REQUIRED SCRIPTS.
From geoNet/examples copygetData.py
event_statsll.py
processData.py
plot_accvel.py
plot_psa.pyto your working directory
2) GET THE DATA FOR THE EARTHQUAKE.
In getData.py set BASE_URL to the location to the geoNet URL e.g
BASE_URL="ftp://ftp.geonet.org.nz/strong/processed/Proc/2017/02_Feb/2017-02-01_102129/Vol1/data/"save and run.
3) GET THE LIST OF STATIONS FOR THE EARTHQUAKE
Run event_statsll.py which creates event_stats.ll and places it
in your working directory.4)PROCESS THE GEONET VOL1 DATA TO GET NORTH, EAST AND VERTICAL COMPONENTS
Run processData.py which filters and rotates the SMS data placing them
in Vol1/data/accBB etc.
Note: It is better to redirect the std output to a file like so
python processData.py > processData_output5) CREATE PLOTS FOR OBSEREVED DATA
Run plot_accvel.py and plot_psa.py
- Ahsan
- Ground motion simulation computation (Sung)
- Upload Rupture model to /nesi/projects/nesi00213/RupModel/Realtime/MonYEAR where MonYear is like Dec2016, Jan2017. Under this directory, you need Srf and Stoch.
- Upload Velocity model to /nesi/projects/nesi00213/VelocityModels/. Currently we have Canterbury or South_Island. Unless it is a Canterbury specific model, place it under South_Island. We will be doing some clean up later.
Your velocity model should include rho3dfile.d, vp3dfile.p, vs3dfile.s, params_vel.py and outputs from gen_cords (gridfile, gridout, model_bounds, model_coords, model_params). - Upload station files to /nesi/projects/nesi00213/StationInfo. This should include geonet_stations_xxx.ll, .vs30 and .vs30ref.
- Go to /nesi/projects/nesi00213/RunFolder and execute ./install.sh. and follow the instruction.
- During install.sh, it will advise you expected wallclock time. Enter a sensible wall clock time.
- Follow the remaining instruction.
- simulation post-processing
- Viktor/Hoby
- General Plotting Instructions:
run on hypocentre, not fitzroy
make sure the scripts are in PATH (hypocentre):
export PATH=$PATH:/nesi/projects/nesi00213/gm_sim_workflow/devel - Plotting Station based data such as IMs, Vs30, Observed PGA, pSA:
plot_stations.py datafile.ll
Run on hypocentre, datafile has 6 line header as described at the top of plot_stations.py - Plotting Timeslice based data (includes PGV, MMI, timeslice animation) on hypocentre:
plot_transfer.py auto /nesi/projects/nesi00213/RunFolder/baes/RT2017Jan17_VMSI_20170117_drill_200m-h0p200_EMODv3p0p4_170116 .
If you have a different username on fitzroy then you will need to add a 4th parameter, your username on fitzroy
will copy the remote fitzroy folder to the current directory (.) and start plotting using defaults
run plot_ts_sum.py to re-run PGV and MMI plots
run plot_ts.py to re-run timeslice image creation
Key lessons from past drills:
March 2017 drill (2014 Mw6.3 Eketahuna Earthquake)
Task | Expected time reqd. | People | Notes |
1. Source (HOBY) | |||
GeoNet Mw email to SRF (a) from Mw tensor get Rup parameters (b) From Rup parameters use createSRF.py to make rupture SRF and to plot (c) Execution of createSRF.py |
1minute
25sec 2min total |
Hoby/Viktor | a) Manually extract location, dip,strike, rake from Mw tensor (Ristau) into setSrfParams.py b) Change TYPE=2 (point source to finite fault) Use either version "3.3" or "5.3.2a" c) Run "createSRF.py" Please note that the execution time of createSRF.py may change based on the event size |
|
|
|
|
2. Velocity Model Domain (ETHAN/ROBIN) | |||
From rupture location to VM dev and domain (a) From location and Mw determine VM origin, 3D extent, and simulation duration (b) Plot VM domain extent (c) Generate VM with H=0.1km for this domain, then extract slices | 1mins 20 mins (on multiple cores) 15 mins for slices 4 mins for H=0.2km VM creation (30 mins for H=0.1km) 20min total
Setup code - 4 mins generateDomain initial (inc. getting CMT data) - 2+2=4 min creating VM (inc. manual adjustment) - 10+1 = 11 genDomain - 6+6+6 = 18
Total ~37-40 minutes
| Ethan, Robin,… |
There was an error in the auto-VM generation on hypocentre which needed to be fixed, and then the code had to be obtained from github again. (Done) (Sung) == items below have beem completed after Feb. drill 1.nx,ny,nzare automatically updated if user manually edits other values like (extent_x,_y,_zmax) 2. if extent_x, extent_y and extent_zmax are not divisible byhh, it automatically increases extent_* to the first divisible value 3. it will automatically work outflo (based on min_vs/(5*hh). The initial value offlo is automated, but user can manually override it by editing params_vel.py (More in item 4) 4.In params_vel,py, extent_x, extent_y, and extent_zmax, sim_duration andflohave automatically generated initial value. User can manually edit params_vel.py to override them, but user MUST run *validate_parameters.py*, which will check the integrity of the parameter file, and adjust automated values such asnx,ny,nzandsufx. This will overwrite params_vel.py (also keeps a backup of old params_vel.py) and show before and after changes side-by-side. (Ethan) Remove topography from preliminary plot to speed up investigateDomain.py Change default CPTs to extend their range
March Drill (I've just put this here for now because I didn't want to delete anything) (Robin) Issues encountered/comments
Overall if everything went smoothly and no manual adjustment was required, this would total something around 15 minutes. (note this is also dependent on the VM domain size/source magnitude.
|
From non-uniform station grid, get statgrid points (a) Locations of statgrid points (b) Locations of GeoNet strong motion stations (c) From NZ Vs30 model get Vs30 values at all points |
(ignore for now) 1-2 hours!! 10mins ~2hours |
Ahsan, Chris |
[need .ll file for all of NZ (Ahsan)] (Done) run statcords (to get station locations corresponding to actual VM domain] |
(a) Plot VM domain extent, stations (statgrid and GeoNet), (b) color based on Vs30 values | 2mins | (a) Viktor (b) Not done | Only done as part of plotting PGA, PGV, PSA, PSA RATIOS Where to get vs30 values? |
3. Observed strong motions (SUNG) | |||
Obtain observations from GeoNet (a) Get recorded motion from GeoNet ftp (b) Process and reformat using defaults (c) Produce plots of acc/vel time series, response spectra (d) Produce IM data files for stations on map plots in GMT (e) Produce Vs30 input files for stations |
| Ahsan | Took 5 mins to create input file for downloading data from geoNet Finished downloading data in 13.1 secs Done Vs30 calcs in 25.0 secs Processing .V1A files ... Done processing in 219.4 secs Creating acc and vel plots for all Observed SMSs ... Done plotting acc, vel in 160.6 secs. Creating pSA plots for all Observed SMSs ... Done pSA plots in 181.2 secs. Writting IMs for GMT plotting ... Done in 32.2 secs. python2 run_event.py 614.36s user 5.28s system 98% cpu 10:32.05 total 10:32 to 11:11 to run post processing code with observations only. code ran in 192.43s, took 5 mins to set the input file. The inputs for this input file and the input file for downloading data are duplicated. using a log scale for plotting observations only IMs on map results in values smaller than 1 to be negative and difficult to interpret. Had to manually adjust cpt color scales for better plots, and manually move gmt plots otherwise they are over written. |
Plot IM values at GeoNet stations on map (for various IMs) | 3 mins | Viktor | Viktor was away, and I (Ahsan) performed this task. It was some what unclear how the GMT plotting scripts (behind the scenes) got its input, e.g the domain boundary. The code ran without problems. There was a need to display the color bar on a log scale, which I was not able to do through the GMT scripts but had to rewrite the natural log of the data instead of the plain data. It was unclear what the log=True option achieves in the python wrapper to the GMT scripts. |
Plot observed vel time series on map | 5mins | Viktor | skipped, need to finish script that does this |
Plot IM values as a function of Rrup and compare with empirical ground motion model (various IMs) | 10 mins | Ahsan | This was still performed with groundMotionSationsAnalysis code by passing Obs data twice (as Obs and Sim data). This task will be performed along side task (c). |
4. Perform GM Simulation (SUNG) | |||
Submit GM Simulation (a) (b) LF sim (queue time + run time) (c) Once LF sim complete, run HF and BB sim |
(a)5mins (b)1:40~3:30 hours (queue+run) (c)13 min |
Sung | (1) We need a standardized event name stored somewhere. (2) File transfer to Fitzroy could be more simplified. Occasionally I had no right permission to access the files generated by other team members. ==> Solution: Each module (VM, Source, Station) can have an "install" script that is executed by the person-in-charge that transfers the necessary files to the appropriate location (derived from the standardised event name) (3) Many codes from various repositories are increasingly becominginter-dependent. We need to make a shared library soon. (4) Wall clock limit estimation is reasonably accurate, with Fitzroy being prone to fluctuate occasionally. (IO suspected). DB needs some tweak to mark outliers.
|
5. Post-processing (VIKTOR) | |||
LF Time slice animation (a) tar, download data from Fitzroy to local (b) From TS produce PGV and MMI plot (c) Produce TS animation | 9 min
|
Viktor | 1) The animation is working fine for single event. However, if there are 2 SRF files in the run, you need to modify the parameter file such that it will generate the animation of the specific event that you want 2) we might want to generate the animation for the multiple events as well?? 3) It would be great if we could include "*modelparams*" in the tar file from Fitzroy, which is needed for (5f) plotting
|
Script preparation and test At SMS locations compare sim vs. obs (a) compare waveforms (b) Compare IMs as a function of Rrup (multiple IMs; sim, obs, empirial) (c) Compare sim, obs, empirical response spectra at individual locations (d) Plot overall bias as a function of vibration period (both sim and empirical) (e) Preparing input for (f) (f) Plot bias(residual) at specific locations on a map | 10mn (a)–(d) 20 mins
(e) 4 mins (f) 5 mins | Hoby
| I have still used Richard code because Daniel/Jason mentioned that they sill have some issue in the std of bias between Obs and empiricalGMM. Once the PP is done there will be significant save in time in terms of file preparation because I will use the same config. as Ahsan
(f) requires modelparamsfile/Srf; need to make sure we have them copied in hypocentrer
|
What to do next
Tasks | Time estimation | |
Hoby | 1) Srf plot location need to be modified 2) 2) | 2days 2days 2-5 days |
Ethan | 1) Streamline the VM generation with gen_coords.csh 2) Write scrip to stitch together multiple PDFs in PDFtk 3) VM domain baseplot to be passed to plot SRF by others | 2days (with other minor tasks) |
Ahsan | (1) The input file for post processing and the input file for downloading data are duplicated. Need to fix this duplication on my part.
| (1) 2 hours |
Sung | The following 3 items are easy to automated, and where we can easily make mistakes.
|
|
Viktor |
| 2.5 days |
Daniel |
|
Feb 2017 drill (2007 Gisborne Earthquake)
Task | Expected time reqd. | People | Notes |
1. Source (HOBY) | |||
GeoNet Mw email to SRF (a) from Mw tensor get Rup parameters (b) From Rup parameters use createSRF.py to make rupture SRF (c) Execution of createSRF.py |
1minute 5mins
10sec 6min total |
Hoby | a) Manually extract location, dip,strike, rake from Mw tensor (Ristau) into setSrfParams.py b) Change TYPE=2 (point source to finite fault) c) Run "createSRF.py" (Make sure to use python2) |
Plotting of SRF (a) SRF plane(s) standard plotting (b) SRF on map to verify location (to come after domain set in step 2) |
3mins 5mins 2mins 10min total |
Hoby |
(a) change manually the SRF and output name in plotSRF.py (Viktor is currently working in improving this part) Execute ‘plotSRF.py’ (Make sure to use python2) (b) plot_srf.py file.srf |
2. Velocity Model Domain (ETHAN/ROBIN) | |||
From rupture location to VM dev and domain (a) From location and Mw determine VM origin, 3D extent, and simulation duration (b) Plot VM domain extent (c) Generate VM with H=0.1km for this domain, then extract slices | 1mins 20 mins (on multiple cores) 15 mins for slices 4 mins for H=0.2km VM creation (30 mins for H=0.1km) 20min total | Ethan, Robin,… |
There was an error in the auto-VM generation on hypocentre which needed to be fixed, and then the code had to be obtained from github again. (Done) (Sung) == items below have beem completed after Feb. drill 1. nx, ny, nz are automatically updated if user manually edits other values like (extent_x,_y,_zmax) 2. if extent_x, extent_y and extent_zmax are not divisible by hh, it automatically increases extent_* to the first divisible value 3. it will automatically work out flo (based on min_vs/(5*hh). The initial value of flo is automated, but user can manually override it by editing params_vel.py (More in item 4) 4.In params_vel,py, extent_x, extent_y, and extent_zmax, sim_duration and flo have automatically generated initial value. User can manually edit params_vel.py to override them, but user MUST run *validate_parameters.py*, which will check the integrity of the parameter file, and adjust automated values such as nx, ny, nz and sufx. This will overwrite params_vel.py (also keeps a backup of old params_vel.py) and show before and after changes side-by-side. (Ethan) Remove topography from preliminary plot to speed up investigateDomain.py Change default CPTs to extend their range |
From non-uniform station grid, get statgrid points (a) Locations of statgrid points (b) Locations of GeoNet strong motion stations (c) From NZ Vs30 model get Vs30 values at all points |
(ignore for now) 1-2 hours!! 10mins ~2hours |
Ahsan, Chris |
[need .ll file for all of NZ (Ahsan)] (Done) run statcords (to get station locations corresponding to actual VM domain] |
(a) Plot VM domain extent, stations (statgrid and GeoNet), (b) color based on Vs30 values | 2mins | (a) Viktor (b) Not done | Only done as part of plotting PGA, PGV, PSA, PSA RATIOS Where to get vs30 values? |
3. Observed strong motions (SUNG) | |||
Obtain observations from GeoNet (a) Get recorded motion from GeoNet ftp (b) Process and reformat using defaults (c) Produce plots of acc/vel time series, response spectra (d) Produce IM data files for stations on map plots in GMT (e) Produce Vs30 input files for stations | (a) to (d) took 7.5 mins. (e) took 40 mins | Ahsan |
Tasks (a) to (d) are automated but (e) is still manual and took the longest time. In (e) the computational time itself only took about 2 mins, it was the formatting that took the most time. domain, no new velocity model was generated. The three key steps (in absence of observed data) were The main takeaway lessons were that |
Plot IM values at GeoNet stations on map (for various IMs) | 3 mins | Viktor | Viktor was away, and I (Ahsan) performed this task. It was some what unclear how the GMT plotting scripts (behind the scenes) got its input, e.g the domain boundary. The code ran without problems. There was a need to display the color bar on a log scale, which I was not able to do through the GMT scripts but had to rewrite the natural log of the data instead of the plain data. It was unclear what the log=True option achieves in the python wrapper to the GMT scripts. |
Plot observed vel time series on map | 5mins | Viktor | skipped, need to finish script that does this |
Plot IM values as a function of Rrup and compare with empirical ground motion model (various IMs) | 10 mins | Ahsan | This was still performed with groundMotionSationsAnalysis code by passing Obs data twice (as Obs and Sim data). This task will be performed along side task (c). |
4. Perform GM Simulation (SUNG) | |||
Submit GM Simulation (a) (b) LF sim (queue time + run time) (c) Once LF sim complete, run HF and BB sim |
(a)5mins (b)1:40~3:30 hours (queue+run) (c)13 min |
Sung | (1) We need a standardized event name stored somewhere. (2) File transfer to Fitzroy could be more simplified. Occasionally I had no right permission to access the files generated by other team members. ==> Solution: Each module (VM, Source, Station) can have an "install" script that is executed by the person-in-charge that transfers the necessary files to the appropriate location (derived from the standardised event name) (3) Many codes from various repositories are increasingly becoming inter-dependent. We need to make a shared library soon. (4) Wall clock limit estimation is reasonably accurate, with Fitzroy being prone to fluctuate occasionally. (IO suspected). DB needs some tweak to mark outliers.
|
5. Post-processing (VIKTOR) | |||
LF Time slice animation (a) tar, download data from Fitzroy to local (b) From TS produce PGV and MMI plot (c) Produce TS animation | 9 min
|
Viktor | 1) The animation is working fine for single event. However, if there are 2 SRF files in the run, you need to modify the parameter file such that it will generate the animation of the specific event that you want 2) we might want to generate the animation for the multiple events as well?? 3) It would be great if we could include "*modelparams*" in the tar file from Fitzroy, which is needed for (5f) plotting
|
Script preparation and test At SMS locations compare sim vs. obs (a) compare waveforms (b) Compare IMs as a function of Rrup (multiple IMs; sim, obs, empirial) (c) Compare sim, obs, empirical response spectra at individual locations (d) Plot overall bias as a function of vibration period (both sim and empirical) (e) Preparing input for (f) (f) Plot bias(residual) at specific locations on a map | 10mn (a)–(d) 20 mins
(e) 4 mins (f) 5 mins | Hoby
| It would be great to have some systematic way to select the "good" waveforms (Probably in the Observation data processing??) and have new station.ll corresponding to these acceptable Obs waveform for validation
(f) requires modelparams file/Srf; need to make sure we have them copied in hypocentrer |
What to do next
Tasks | Time estimation | |
Hoby | 1) Include ground motion duration in the PP plotting 2) Add the empirical ground motion duration 2) get plotSRF improved | 2days 2days 2-5 days |
Ethan | 1) Streamline the VM generation with gen_coords.csh 2) Write scrip to stitch together multiple PDFs in PDFtk 3) VM domain baseplot to be passed to plot SRF by others | 2days (with other minor tasks) |
Ahsan | (1) automate process to produce Vs30 files and their formatting. (2) Produce plots that need Rrup within the existing geoNet real-time processing scripts as opposed to separately running groundMotionStationsAnalysis code separately. (4) Incorporate change to write custom header to .000 .090 .ver GP format files. Issue brought up by Chris McGann | (1) 2 hours (2) 2 hours (3) 2 hours (4) 2hours |
Sung | The following 3 items are easy to automated, and where we can easily make mistakes.
|
|
Viktor |
| 2.5 days |
Daniel |
|
Jan 2017 drill (Lake Grassmere Earthquake)
Task | Expected time reqd. | People | Notes |
1. Source (HOBY) | |||
GeoNet Mw email to SRF (a) from Mw tensor get Rup parameters (b) From Rup parameters use createSRF.py to make rupture SRF (c) Execution of createSRF.py |
1minute 5mins
10sec 6min total |
Hoby | a) Manually extract location, dip,strike, rake from Mw tensor (Ristau) into setSrfParams.py b) Change TYPE=2 (point source to finite fault) c) Run "createSRF.py" |
Plotting of SRF (a) SRF plane(s) standard plotting (b) SRF on map to verify location (to come after domain set in step 2) |
3mins 5mins 2mins 10min total |
Hoby |
(a) change manually the SRF and output name in plotSRF.py (need to be improved) Execute ‘plotSRF.py’ (b) plot_srf.py file.srf |
2. Velocity Model Domain (ETHAN/ROBIN) | |||
From rupture location to VM dev and domain (a) From location and Mw determine VM origin, 3D extent, and simulation duration (b) Plot VM domain extent (c) Generate VM with H=0.1km for this domain, then extract slices | 1mins 20 mins (on multiple cores) 15 mins for slices 4 mins for H=0.2km VM creation (30 mins for H=0.1km) 20min total | Ethan, Robin,… |
|
From non-uniform station grid, get statgrid points (a) Locations of statgrid points (b) Locations of GeoNet strong motion stations (c) From NZ Vs30 model get Vs30 values at all points |
(ignore for now) 1-2 hours!! 10mins ~2hours |
Ahsan, Chris |
[need .ll file for all of NZ (Ahsan)] (Done) run statcords (to get station locations corresponding to actual VM domain] |
(a) Plot VM domain extent, stations (statgrid and GeoNet), (b) color based on Vs30 values | 2mins | (a) Viktor (b) Not done | Only done as part of plotting PGA, PGV, PSA, PSA RATIOS Where to get vs30 values? |
3. Observed strong motions (SUNG) | |||
Obtain observations from GeoNet (a) Get recorded motion from GeoNet ftp (b) Process and reformat using defaults (c) Produce plots of acc/vel time series, response spectra | 45 mins total for (a) and (b) 60 mins for generating Vs30 values. (one-off task) 5 mins for (c) | Ahsan |
Majority of time spent trouble shooting. See comment below GeoNet people make errors which can cause errors and breakdown our code. The data location has files that should have been removed. Take a look at ftp://ftp.geonet.org.nz/strong/processed/Proc/2013/08_Aug/2013-08-16_023105/Vol1/data . Files ending with _ should not be there and nor should V2A and V3A data be there. You will also see three stations named like Q13924J03.V1A that don't follow the naming convention used for all the other data files. This broke down my code. For the next drill, I will change the code to catch these as exceptions and continue smoothly. |
Plot IM values at GeoNet stations on map (for various IMs) | 3 mins | Ahsan | Calculations of PGVs, PGAs and pSA for set of chosen periods took about 3 mins, these were then passed on to Viktor who plotted them. |
Plot observed vel time series on map | 5mins | Viktor | skipped, need to finish script that does this |
Plot IM values as a function of Rrup and compare with empirical ground motion model (various IMs) | 10 mins | Ahsan | Code ran as expected. |
4. Perform GM Simulation (SUNG) | |||
Submit GM Simulation (a) (b) LF sim (queue time + run time) (c) Once LF sim complete, run HF and BB sim |
(a)5mins (b)5 hours (queue+run) (c)13 min |
Sung |
gen_ts was broken (fixed) Will be nice to have auto file transfer (in 3 separate streams esp. for VM model) Will need to separate HF from BB and run it alongside LF.
|
5. Post-processing (VIKTOR) | |||
LF Time slice animation (a) tar, download data from Fitzroy to local (b) From TS produce PGV and MMI plot (c) Produce TS animation |
10mins 25mins 25min
|
Viktor |
needed to fix up scripts to work with new parameter style mainly needed to fix script to use new parameter style, should run automatically after step A need to automate a little more, run time ~5 mins |
Script preparation and test At SMS locations compare sim vs. obs (a) compare waveforms (b) Compare IMs as a function of Rrup (multiple IMs; sim, obs, empirial) (c) Compare sim, obs, empirical response spectra at individual locations (d) Plot overall bias as a function of vibration period (both sim and empirical) (e) Preparing input for (f) (f) Plot bias(residual) at specific locations on a map | 10mn (a)–(d) 30 mins
(e) 4 mins (f) 5 mins | Hoby
(f)Viktor |
default parameters used, runs in 2 mins but needed to fix a few things up |
What to do next
Tasks | Time estimation | |
Hoby | 1) Include ground motion duration in the PP plotting 2) Add the empirical ground motion duration 2) get plotSRF improved | 2days 2days 2-5 days |
Ethan | 1) Streamline the VM generation with gen_coords.csh 2) Write scrip to stitch together multiple PDFs in PDFtk 3) VM domain baseplot to be passed to plot SRF by others | 2days (with other minor tasks) |
Ahsan | (1) GeoNet people make errors which can cause errors and breakdown our code. The data location has files that should have been removed. Take a look at ftp://ftp.geonet.org.nz/strong/processed/Proc/2013/08_Aug/2013-08-16_023105/Vol1/data . Files ending with _ should not be there and nor should V2A and V3A data be there. You will also see three stations named like Q13924J03.V1A that don't follow the naming convention used for all the other data files. This broke down my code. For the next drill, I will change the code to catch these as exceptions and continue smoothly. (2) Deal with new stations better. Currently I maintain a list of about 1350 stations and new stations coordinates are read from there. But in this drill one new station was not in that extensive list. A better solution would be to read the data file and extract the coordinates rather than relying on a list. (3) I shall add Vs30 = 500 for any new stations and give the list of new stations to Kevin after our drills. He can then provide a better Vs30 estimate. This way we simply the problem of knowing Vs30 for new stations. What do you propose? These are not big issues in general but they are if we want unbroken, smooth work flow during real time simulation. I shall add these to my task list. | (1) 2 hours (2) 5 hours (3) 2 hours |
Sung | The following 3 items are easy to automated, and where we can easily make mistakes.
|
|
Viktor |
| 2.5 days |
Daniel |
|
Dec 2016 drill (Wilberforce Earthquake)
Task | Expected time reqd. | People | Notes |
1. Source (HOBY) | |||
GeoNet Mw email to SRF (a) from Mw tensor get Rup parameters (b) From Rup parameters use createSRF.py to make rupture SRF (c) Execution of createSRF.py |
1minute 5mins
10sec 6min total |
Hoby | a) Manually extract location, dip,strike, rake from Mw tensor (Ristau) into setSrfParams.py b) Change TYPE=2 (point source to finite fault) c) Run "createSRF.py" |
Plotting of SRF (a) SRF plane(s) standard plotting (b) SRF on map to verify location (to come after domain set in step 2) |
3mins 5mins 2mins 10min total |
(a) Hoby (b) Viktor | Execute ‘plotSRF.py’ (select title for top of script) (a) Hoby : Just needed to change SRF name (b) Viktor: make 2 plots on 1 image, 1 zoomed in with slip visible, 2 whole of NZ showing outline/hypocentre Finished automating. only SRF needed, no reliance on params.py etc... |
2. Velocity Model Domain (ETHAN/ROBIN) | |||
From rupture location to VM dev and domain (a) From location and Mw determine VM origin, 3D extent, and simulation duration (b) Plot VM domain extent (c) Generate VM with H=0.1km for this domain |
2mins
1mins 15 mins (on multiple cores) 20min total | Ethan, Robin,… | 1) Origin lat/lon/depth written into 'generate_parameters.py' python script 2) Run 'generate_parameters.py', output is a text file which is read by the next step 3) Run 'genDomain.py' this generates the velocity model, extracts slices. Stores the results in Rapid_Model Need to implement duration calculations Need to streamline Gen_Coords to sync with model parameters Run gen_cords.csh and plot Create .p/s/d files |
From non-uniform station grid, get statgrid points (a) Locations of statgrid points (b) Locations of GeoNet strong motion stations (c) From NZ Vs30 model get Vs30 values at all points |
(ignore for now) 1-2 hours!! 10mins ~2hours |
Ahsan, Chris |
[need .ll file for all of NZ (Ahsan)] (Done) run statcords (to get station locations corresponding to actual VM domain] |
(a) Plot VM domain extent, stations (statgrid and GeoNet), (b) color based on Vs30 values | 2mins | (a) Viktor (b) Not done | Where to get vs30 values? |
3. Observed strong motions (SUNG) | |||
Obtain observations from GeoNet (a) Get recorded motion from GeoNet ftp (b) Process and reformat using defaults (c) Produce plots of acc/vel time series, response spectra | 40 mins total | Ahsan | Uploading to cloudup was tricky as the files need to be downloaded from the linux box first. |
Plot IM values at GeoNet stations on map (for various IMs) | 1 hour (GMT) | Ahsan | updating e3d.par domain info |
Plot observed vel time series on map | 5mins | Viktor | need to automate location of stations involved, zoom levels etc... code exists, needs to be brought together / cleaned up |
Plot IM values as a function of Rrup and compare with empirical ground motion model (various IMs) | 20min 1hour 30mins | Ahsan | Richard's code runs for 3 mins. Had to go through a number of iterations to get nice looking plots. Has a good idea to reduce the time significantly. Knows what parameters to change. Not easy to automate. |
4. Perform GM Simulation (SUNG) | |||
Submit GM Simulation (a) prepare input params.py, pointing to SRF, VM, and stations of interest etc. (gen_cords.sh and statlist2gp.py need to be run) (b) LF sim (queue time + run time) (c) Once LF sim complete, run HF and BB sim |
20mins 512 cores (8node) [wait+run 2hours] 30mins 4hours |
Sung |
Good opportunity for automation. (a) can be done in a sec. |
5. Post-processing (VIKTOR) | |||
LF Time slice animation (a) tar, download data from Fitzroy to local (a) Produce TS animation (b) From TS produce PGV plot (c) Produce MMI plot from PGV |
30mins 30mins 5min 5min
|
Viktor |
(can be streamlined, at later date) vm domain used by default, determine if regional sites should be added (crowding), padding around vm based on rot, domain if rotation small mostly automated TS animation, some parameters needed such as colour range single script does both PGV, MMI - need to finish automating colour range for PGV |
Script preparation At SMS locations compare sim vs. obs (a) compare waveforms (b) Compare IMs as a function of Rrup (multiple IMs; sim, obs, empirial) (c) Compare sim, obs, empirical response spectra at individual locations (d) Plot overall bias as a function of vibration period (both sim and empirical) (e) Preparing input for (f) (f) Plot bias(residual) at specific locations on a map | 10mn (a)–(c) 30 mins
(d) 20mn
(e) 10 mins | Hoby |
make a script that plots just this and loops over period values - done self contained input file used and model_params |
What to do next
Tasks | Time estimation | |
Hoby | Fix bias plotting for both empirical and physics-based models | 2days |
Ethan | 1) Streamline the VM generation with gen_coords.csh 2) Write scrip to stitch together multiple PDFs in PDFtk 3) VM domain baseplot to be passed to plot SRF by others | 2days (with other minor tasks) |
Ahsan | 1) Update all_geoNet_stations.ll to include SMS that closed in 2010. This will add more stations to the list and we won't have a problem when dealing 2) Some python scripts (code that downloads data, code that processes data) that run one after another can and will be automated. | 5days |
Sung | The following 3 items are easy to automated, and where we can easily make mistakes.
|
|
Viktor |
| 1 - 2 days |
Daniel |