Adding GUI System Tests - #249
Conversation
…m-tests and --run-unit-tests
… project in testing
… due to matlab runner not exiting properly
StephenNneji
left a comment
StephenNneji
left a comment
There was a problem hiding this comment.
Thanks for this, please see comments
Also please update the unit testing section in contributing.md appropriately
|
|
||
|
|
||
| def pytest_addoption(parser): | ||
| parser.addoption("--run-system-tests", action="store_true", default=False, help="Run GUI system tests offscreen") |
There was a problem hiding this comment.
In my testing "--run-system-tests" and "--run-system-tests-show"", both show the ui, are both needed?
| run: xvfb-run pytest -s tests/ ${{ inputs.pytest-options }} --cov=rascal2 --cov-report=term | ||
| - name: Pytest System Tests | ||
| shell: bash -l {0} | ||
| run: xvfb-run pytest -s tests/system/ ${{ inputs.pytest-options }} --cov=rascal2 --cov-report=term |
There was a problem hiding this comment.
This line is missing the system-test flag.
I think since the system tests are not expensive, maybe we should run them by default i.e pytest tests should run unit and system
then the provided flags can be to skip instead of run i.e --skip_system_tests. This will also be neater on the ci so we have a single test run and coverage for all the test..
| def tearDown(self) -> None: | ||
| super().tearDown() | ||
|
|
||
| def test_main_window(self): |
There was a problem hiding this comment.
please delete test_ui.py as it duplicates this
2 participants
This PR introduces a new form of GUI System Test which creates an instance of RasCal2, performs a sequence of events (clicks, typing, etc) and tests the validity of the output. This is more useful than isolated unit tests as it tests theinteraction between different parts of the program and more closely resembles a realistic user workflow.
System tests inherit the
GuiSystemBaseclass, which handles thesetUpand tearDownof theself.main_window` instance, simplifying how the tests are written.Unit tests and System tests are differentiated in pytest. We can run only unit tests via:
pytest testsorpytest tests --run-unit-testsSystem tests are run with the window hidden by default and can be ran via:
pytests tests --run-system-teststo show the window while tests are running:
pytests tests --run-system-tests-showIf we want to include screenshot/eyes tests in the future, a flag can be created for them to differentiate screenshot tests from unit and system tests.