You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 17 Next »

IM Calculation Improvement

Submission integrated into automatic workflow

Large simulations chunked during loading and then processed. Memory usage is higher than expected but do not envision any cases where the memory usage exceeds limitations.

 

Management DB

Improved speed to return jobs needing to be submitted

Added queue, start and end time parameters to be stored in the DB.

Added retries count to automatically retry simulations

Daemon

Adjustable wall-clock parameter

18p9 run

Selected the runs to be done

Hossack 501r
PaeroaC 167r
Mangatete 334r
AlpineF2K 1r

 

Testing Plan

Day 1:

  • Generate list of faults

  • Autogenerate SRF and VM

Day 1 (night):

  • First run of 18p9

Day 2

  • Fix automation steps

Day 2 (night)

  • Second test run

Day 3

  • Repeat as necessary

Day n (waiting for storage issues to be fixed)

  • Port workflow to Maui

Day n + 2

  • Run 18p9 test on Maui

  • Fix automation as necessary

Day n + X

  • Resubmit 18p6 magnitude faults on Maui

 

 

Testing Results

Day 1
  • Remove dependancy from SRF_generation on post-processing (change to empirical engine)
  • Permission issue on VMs generated
Day 2
  • VM code missing plotting data and incorrect QCoRE repository location – ensure QCoRE is configured correctly
  • HF issue - issue with random seed / -i parameter
Day 3
  • BB - DT/NT relation between HF and LF inconsistent resulting in failing BB
    • Missing parameter in the vm-generation step. Going to make generation scripts / parameter checks before running.
  • Retry parameter not functioning correctly - now logging into the management error log on each retry and each reset of the retries
  • IM calc tasks:
    • Completion checking
    • Automation
    • Inconsistent naming
Day 4
  • Task time missing log - functionality existed but did not call the functions.
Test - Missing Batch 1 - Cybershake 18p6

This batch was missed because of an issue with large binary files - these submissions were moved from the automatic submission and weren't run.

  • IM Calc not updating to in-queue in DB creating lots of repeated submissions of IMCalc job. (fixed)

Was plotted manually for validation.

Test - Wrong Magnitude - Cybershake 18p6

SRF / VM Generated

  • Determined faults that didn't have VMs generated and found the issue

Simulations Run

  • 1 LF simulations had errors - potentially IO error (re-running fixed this)
  • IM Calc marked as failed but completed. (fixed)
  • Need to fix error logs
  • Only some tasks automatically retried

Viewing output now

Downloading IMs and plotting on hypocentre

 

Outstanding Tasks

Permission issue on VMs generated

Automatic plotting of single realisation for each fault

Cleanup of intermediary steps in workflow

Slurm script generation for SRF/VM plotting on Mahuika (including SRF and VM plots)

  • No labels