The purpose of this document is to describe the standards and procedures to follow during the software testing phases of the QuakeCoRE ucgmsim git repositories.
1. Scope
2. Test priorities
(1) Prioritize GitHub repositories for testing
Identify in-use/abandoned repositories
Identify type of individual application (script/library)
Different type of application has different testing strategy
Count usage frequency of individual applications
Summary: https://docs.google.com/document/d/1ntFrNsMBo1G5bqTCnJIYVsTiuLApUbrf4vNyYMdI298/edit
3.Test plan
(1) Test method:
Black-box: testing without knowledge of the code, compare if the test output is the same as the sample output; often used in testing scripts
White-box: testing with knowledge of the code, often used in unit test for library.
(2) Test framework:
- Pytest
(3) Test design:
Identify type of the application: Script: tested as a whole; Library: unit testing
Identify input and output: what is used to test; what is produced from the test; contact the developer if confused
Formulate tailored testing plan for individual application if needed: eg. design output folder structures, writing config/readme files
Select benchmark/correct sample output
Write testing script for the application
Execute the test
Compare test output with sample output
(4) Test execution
(5) Test report
Examples
(1) How to identify the type of application and corresponding test method
(2) How to design the structure of a testing folder
