URL: https://github.com/ucgmsim/Pre-processing
Summary
Everything related to producing input for GMSim (NonUniformGrid, SrfGen, geoNet), including a wrapper for VM generation that calls Ethan's VM binary code, which uses SRF info file as an input.
Current Status : Working
SRF and VM are generated directly from NHM and GCMT by a single Slurm script. Tested with CS18p6 pre-processing and works fine.
SRF generation uses 80 concurrent processes (Hyperthread enabled + Python multiprocessing)
VM generation uses 16 threads for each VM with 5 VMs produced in parallel. (2.5 times after than no hyperthread)
Known Issues
Desirable to have a checkpointing.
- SRF checkpointing: Only needs to skip completed faults
- VM checkpointing: As it depends on SRF info files in autosrf, best to create a temporary copy of autosrf that only contains faults whose VMs are incomplete/missing (Only needs to copy SRF info). Each resume will produce vminfo.csv files containing successfully produced VM meta data, vminfo.csv should be different name if it is a resume to avoid overwriting. Extra routine may be added to merge vminfo files.