Extend testing framework

[yannickl96]

The idea of using sub-FSMs with a common interface for single tests is nice. It would be even cooler to extend that thought to parametrize these sub-FSMs with input and expected output values.

What I had in mind is:

• Extend the TestHandler interface to include a method Bool success() which indicates if the test was a success
• Provide a default module ``mkTestCheckExpected(#parameter t input, #parameter t2 expectedOut) (TestHandler) which extends the currently implemented functionality of mkTestsMainTest by a check if the provided output of the dut matches the parameter expectedOut
• Create a module mkDashBoardTestbench which uses the mkTestCheckExpected module and keeps track of successful and failed tests and displays the total statistics of totally passed tests.
• Extend the documentation to showcase an example of the new functionality.

The main idea is to provide a basic functionality similar to assertEquals in programming so it's easy to generate testcases and see if the latest commits to a project actually improved the code (i.e. more tests passing).

[jahofmann]

Such a feature was always on "my list" but I never got around to it. My plan was to use the ModuleContext features of Bluespec to annotate "Testing" modules that are automatically enumerated and run when compiling with e.g. make test and use some kind of assert functions to check for failures. Using BlueTCL one could even get the annotated modules programmatically to enable e.g. parallel simulation. In the end it would look a bit like tests in rust https://doc.rust-lang.org/book/ch11-01-writing-tests.html

[yannickl96]

Just to make sure that I understand what you think: You would implement it like

module [BSVToolsTest] mkTestcase(TestHandler);
......
endmodule

And each module of type BSVToolsTest (name is just an example) would collect test information via ModuleCollect?
The bracket notation is what I understand under ModuleContext, or do you think of something else?

[jahofmann]

Yes, exactly.

[yannickl96]

Feel free to assign me. I'd love to implement this in a synthesizable fashion.

[yannickl96]

Just some nice to have: Output to an XML file in JUnit format, so test reports could be viewed after using Gitlab CI.