View Source

This page summarizes the archiving of some of the current CyberShake results.

	Source	VM (.cfgs) / vm_params.yaml	BB	IM
20p9	Dropbox (.csv/.info)	Dropbox	Dropbox (2.1Tb)	/isilon/CS_ims/
20p6(=20p4_200)	Same as 20p4 /isilon/cybershake/v20p4/Sources	Dropbox Isilon: /isilon/cybershake/v20p4/200m_vmcf	Dropbox (8.0Tb)	IM_calc – Dropbox /isilon/CS_ims/
20p5	Dropbox (.csv/.info)	Dropbox	Dropbox	/isilon/CS_ims/
20p4	Dropbox (.csv/.info) Isilon: /isilon/cybershake/v20p4/Sources (full)	Dropbox /isilon/cybershake/v20p4	/nesi/nobackup/nesi00213/archive/Cybershake (11Tb) Dropbox	/isilon/CS_ims/
19p5	Full Srf/Stoch v19p5_sources – Dropbox (766Gb)	Dropbox (vm_params)	Dropbox (11Tb)	/isilon/CS_ims/
18p6	Full Srf/Stoch v18p6_sources – Dropbox (846Gb)	Dropbox	Nearline /nesi/nearline/nesi00213/RunFolder/to_nearline_18p6/ v18p6_waveform_backup.tar (Can't retrieve from it) (43.67Tb)	Dropbox (IMDB.h5) /isilon/CS_ims/
18p5			Isilon: /isilon/cybershake/v18p5
18p4			Isilon: /isilon/cybershake/v18p4
17p9			Isilon: /isilon/cybershake/v17p9

Note:

For a Cybershake version (eg. v20p9), we keep 3 items

Sources: .csv and .info per fault. All files packed into one tar file. v20p9_Sources.tar . (pre-v20p4, we kept all Srf and Stoch files)
VMs: Only nzvm.cfg per fault. All files packed into one tar file. v20p9_VM_cfgs.tar
BB: BB.bin files for all realisations packed into a tar file for each fault. (eg. AlpineF2K.tar)

An automated archive script can package all CS files and generate a Dropbox upload command that can be executed from NeSI login node (in "screen" session)

Dropbox has a maximum file size of 350Gb. The auto script will split the TAR files into multiple parts of 340Gb size. To join them (before extracting from TAR),

cat ${TARNAME}.parta* > ${TARNAME}.joined
tar xvf ${TARNAME}.joined