Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Expand
titleFmax

Fmax calculation:

Still to be intergrated into the workflow
Currently have a converted matlab script into python that is hacked together, but needs work on intergration.

Extra steps:

  • Fmax output column rename to stem of the mseed file (instead of ev_sta_chan, have a record_id column)
  • Add a skipped records csv and list reasons why the snr file was skipped for fmax calc
  • Add magic numbers to the config

I'd say the steps for this also need to be investigated and approved by the wider team (later work to be done)


Expand
titleGMC

GMC:

Blackbox, just requires mseed files
Outputs a gmc_predictions.csv file that goes into the flatfiles directory

TODO

Make work with the full workflow (Upgrade to tensorflow2)
GMC station value is randomly the channel?
Ensure proper pre-processing is done on waveforms for determining fmin


Expand
titleProcess

Process records:

Takes a fmax file and the gmc predictions for fmin and fmax and requires mseed files

A quick check is done that there is all 3 components (This is some bug that needs to be fixed in geonet.py)

Initial pre_processing (Same as whats written in SNR) is then performed and processing is skipped if the Inventory failed to fetch or failed to remove sensitivity

fmax and fmin values are then extracted from the gmc and fmax dataframes
For fmin the max fmin_mean value is taken (max across all 3 components)
fmax is the min of all 3 components

If these values are not found then for the fmin, lowcut is set to 0.05 and if fmax is missing then the highcut is set to 1 / (2.5 * dt)
A simple check is performed that the lowcut is not higher than the highcut, if so the record is skipped

Then the 000, 090 and ver components are selected from the initial processed mseed (rotated)
Tries to grab NE if not then tries XY and if not then skips record

Then applies the following for processing

  • butter bandpass filter - order of 4 fs = 1 / dt
  • Remove zero padding
  • Calculate velocity 
    vel_000 = integrate.cumtrapz(y=acc_bb_000, dx=dt, initial=0.0) * g / 10.0
  • Calculate displacement 
    disp_000 = integrate.cumtrapz(y=vel_000, dx=dt, initial=0.0)
  • Fit a six-order polynomial 

    coeff_000 = np.polyfit(np.arange(len(disp_000)), disp_000, poly_order)
  • Find the 2nd derivative of the coefficients 
    coeff_000_2nd = np.polyder(coeff_000, 2)
  • generate polynomial from the coefficients
    poly_000 = np.polyval(coeff_000_2nd, np.arange(len(acc_bb_000)))
  • Subtract the polynomial from the original acc series

After all that saves as individual component files named the same as the mseed but with the extentions of .000 .090 and .ver in the waveforms directory but under "processed"