Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement unit and system test infrastructure #17

Open
4 tasks
harrisonCassar opened this issue Aug 1, 2020 · 0 comments
Open
4 tasks

Implement unit and system test infrastructure #17

harrisonCassar opened this issue Aug 1, 2020 · 0 comments
Assignees
Labels
enhancement New feature or request high priority Extra attention is needed input wanted Issues whose implementations are open for discussion and suggestions
Milestone

Comments

@harrisonCassar
Copy link
Contributor

Larger coding projects implement various forms of continuous integration (CI) as a means of ensuring that all code modifications and enhancements, before being integrated into the master code repository, are correct and robust. Although the SPARTAN project may not be as big as other "larger" coding projects, implementing a "miniature" form of CI would be most certainly ideal, as this ensures that as the SPARTAN FSW and sensor driver library develops further and continues to implement support for more and more sensor hardware modules, the integrity of our existing code-base is preserved.

When would this testing infrastructure be run? Ideally, before any pull request is accepted and merged with master, all code modifications on that development branch would run these unit and system tests with the current master branch already merged in. If these checks pass, and no problems are found by the dedicated software reviewers, then we should be confident in our ability to accept the PR and merge into master. Defining how this test infrastructure would exactly look like is completely up to us. Some ideas of how I was thinking the implementation would look like follows:

  • We could have two sub-directories within the Software/test directory, one for unit_tests, and the other for system_tests. If we decide that separating the two types of tests in our context is too much, then we can leave this distinction out.
  • A clear and user-friendly infrastructure for adding tests is a MUST. Ideally, before accepting and merging any source code modifications and enhancements, tests should also be added by the developer to confirm the modifications correctness, as well as ensure future modifications do not unintentionally harm the validity of these to-be-merged changes. Something potentially along the lines of having the previously mentioned system_tests directory have sub-directories, one for each test. Everything needed for that test (with the exception of SPARTAN source code) would be contained within its corresponding sub-directory.
  • Make sure the testing infrastructure is MODULAR and easy to automate. We want to be able to easily run these tests without much effort. Potentially this means mandating that each test has some run or input shell script that will run the built SPARTAN executable in the way needed to test the modification-of-interest.
  • A dedicated make target (such as make test) would build the SPARTAN FSW and automatically run all tests, likely through calling some testing script. If implementing the previously mentioned layout of system test "sub-directories", then this testing script would have to simply loop through all available sub-directories within the test directory and call their respective run or input scripts. If any tests fail, the user will be notified automatically, and provided more information for means of debugging.

These are just some rough ideas I was thinking of when first laying out this issue... Feel free to work with me and everyone else on the development team to make this into something amazing!

Related to #10 , as we should have a section describing the testing infrastructure's layout, as well as both the required and optional interfaces that developers need to know when adding new tests (i.e. must have run script, optional options.txt file that describes extra options to pass to SPARTAN, etc.).

@harrisonCassar harrisonCassar added the enhancement New feature or request label Aug 1, 2020
@harrisonCassar harrisonCassar added this to the Fall 2020 milestone Aug 1, 2020
@harrisonCassar harrisonCassar added the high priority Extra attention is needed label Aug 1, 2020
@harrisonCassar harrisonCassar added the input wanted Issues whose implementations are open for discussion and suggestions label Sep 26, 2020
@kevinl120 kevinl120 self-assigned this Nov 23, 2020
harrisonCassar added a commit that referenced this issue Jan 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request high priority Extra attention is needed input wanted Issues whose implementations are open for discussion and suggestions
Projects
None yet
Development

No branches or pull requests

2 participants