GitHub URL: https://github.com/ucgmsim/qcore
What is this repo about?
Commonly used functions that can be shared by code from different repositories.
Repo status
| README present | No |
| Is Public? | Yes |
Number of commits | 789 |
Last time Updated | April 4 2019 |
Functionalities
- Description: State how this function is used or interacts with other sw components.
- Status: (1: not working, 2: unstable, 3: works under specific condition, 4: works with known issues, 5: perfect )
- Tests: (1: none, 2: broken/outdated. 3: with limited coverage, 4: works with known issues, 5: perfect)
- Doc (1: none, 2: outdated, 3: with limited coverage, 4: mostly ok, 5: perfect) Give a link
- Frequency of use: Daily, Weekly, Monthly, Yearly, Never
- Frequency of code/req. change: Daily, Weekly, Monthly, Yearly, Never
- Bus Factor: Number of people that are familiar with the code (1-7)
| Functionality | Description | Status | Known issues | Tests? | Doc? | Frequency of use | Frequency of code/req. change | Bus Factor |
|---|---|---|---|---|---|---|---|---|
| setup.py | Can install itself under site_packages | 5 | 1 | 1 | Daily - Weekly | Monthly-Yearly | 7 | |
config.py | Determines the machine type and loads its associated configuration in HPC environment. (Even if it is not Maui/Mahuika it loads a default configuration) | 5 | 3 (self evident) | 7 | ||||
| binary_version.py | Returns the path of the executables for a given version specified in the config | 5 | 3 (comments) | 7 | ||||
| constants.py | Default constants | 4 | Work in progress. More constants can be added | 3 (comments) | 7 | |||
| empdb.py | From an HDF5 formatted empdb,
| 5 | 1 | 3 (comments) | 2 | |||
| formats.py | Misc. functions/classes related to data loading
| 5 | For what it is, it appears ok (but this is better be expanded or relocated) | 3 (comments) | 7 | |||
| gen_coords.py | Generate coordinate files for an emod3d domain (set of grid points) | 4 | params_vel.py deprecated. Move to Pre-processing. | 3 | 3 (comments) | Monthly (srfinfo2vm and nhm2vm) | 7 | |
| geo.py | A set of functions for geometric calculation and conversions | 4 | The code is mature and well-tested, but use of docstring for comment desirable. | 4 | 3 (comments) | Weekly (Pre-processing, GMSimViz, xyts..) | 1 | |
| gmt.py / gmt_test.py | GMT wrappers | 4 | grid dx and dy should be by default automatically calculated consistency among accessory functions' working directory logic add support for different interpolation methods (xyz2grd, surface, nearestneighbour etc...) avg_ll calculated elsewhere should be local function that works over equator | 3 | 3 (comments) | Weekly | Weekly | 2 |
| im.py | Load a csv file containing IM data and reorder | 4 | Could merge with the load_im_file() in formats.py | 1 | 3 (comments) | Weekly (slurm_gm_workflow & Empirical_Engine) | Never | 5 |
| imdb.py | IMDB access api that was created by imdb_create.py | 4 | IMDB format may be subject to change. | 1 | 3 (comments) | Daily (Seisfinder2, gm_selection, ground_failure) | Weekly | 3 |
| mpl.py | Functions related to Matplotlib plotting | 5 | (code updated a year ago but used daily : must be solid) | 1 | 3 (comment) | Daily (Seisfinder2, visualisation, techplatform2, pwavemethod) | Yearly | 4 |
| nputil.py | argserach: find the index of elements of one array in another array if found | 5 | Solid, but if the example from stack overflow is added, it will be easier to understand. Should it be here? | 1 | 3 (comment) | Weekly (validation, visualisation) | Yearly | 7 |
| pool_wrapper | Wrapper to facilitate multiprocessing | 5 | Solid | 1 | 1 (not even comment) | Weekly (IM_calculation, Pre-processing) | Yearly | 7 |
| shakemap_grid.py | Generates shake-up header using given values for PAGER | 4 | Solid, but should it be here? | 1 | Rarely | 2 | ||
| shared.py | Miscellaneous generic functions | 4 | There are other function/scripts of the same name in GroundFailure and createSRF slurm_gm_workflow has a duplicate shared.py is_virtual_station : can be removed. Not used by other un-archived repos except visualisation | 1 | Weekly get_stations: slurm_gm_workflow get_corners: visualisation is_virtual_station: Only at visulisation
| Monthly | 7 | |
| simulation_structure.py | Constructs standard path for simulation structure | 5 | 1 | slurm_gm_workflow Pre-processing GroundFailure | Weekly | 7 | ||
| siteamp_models.py | site amplification func. | 5 | Mature Move to slurm_gm_workflow | 1 | 3 (comment) | Weekly (slurm_gm_workflow - bb_sim) | Weekly | 1 |
| sosfiltfilt.py | Required functions for sosfiltfilt to work with scipy 0.16+ copied from scipy. | 4 | Not needed anymore. qcore.timeseries.py can directly import from scipy.signal | 1 | 3 (comment) | Weekly | Yearly | 1 |
| srf.py | Various functions related to loading srf file and extracting info from it. | 5 | Last update was in Feb after frequent updates - should be stable | 3 | 3 (comment) | Weekly (slurm_gm_workflow, visualisation, im_calc) | Weekly | 5 |
| testing.py | Common testing boilerplate codes | 5 | (Tested extensively by Travis integration) | 5 | 3 (comment) | Daily | Monthly | 2 |
| timeseries.py | Waveform time-series related functions | 4 | Binary workflow is solid. Some functions are old, but still used. | 1 | 3 (comment) | Daily | Monthly | 3 |
| utils.py | Misc generic functions | 4 | Remove DotDictify load_params yams related functions can be separated update function should be defined inside load_sim_params(), no need to be visible. | 1 | 3 (comment) | Weekly (load_sim_params) | Monthy-Never | 7 |
| validate_vm.py | VM validation | 5 | 1 | 3 (comment) | Weekly-Monthly (Pre-processing, slurm_gm_workflow) | Monthly | 2 | |
| xyts.py | Loads xytx.e3d file, extract info out of it. | 5 | 3 | 3 (comment) | Weekly-Monthly (visualisation, GMSimViz, | Yearly | 1 |
Suggested Improvements / New Features
- Description: State how/why this will be useful
- Timeline: Estimate of how many sprints will it take to develop
| Functionality | Description | Timeline |
|---|---|---|
| empdb.py | More features to be added after empdb format is stable | |
| im.py and formats.py to be merged | Merge and call it imutil.py to contain all raw IM related functions | |
| shake-up_grid.py | How should we incorporate PAGER into our workflow | |
| gen_coords.py | Move to Pre-process | 1 day |
| siteamp_models.py | Move to slurm_workflow | 1 day |
| QCore restructuring | Need more coherent structure. eg. IM related functions all under IM.py (or IMutil.py) | |
| Better API doc | SRF, IMDB related libraries deserve better API doc. |