Verification and Validation Toolkit

By @dabecart.

An easy to use application to define a series of test cases to verify and validate your project, be it hardware or software.

Each test case will be comprised of a call to your tool to test a part of your project. When your tool runs, it's expected to generate some sort of output (at the moment, only console output is supported). On the V&V toolkit you will select the "important" parts of this output, and that will be the result of the test case. When all test cases are run, you'll be able to generate a report of the device. This will be the expected output to be generated by other copies/versions of your project.

Finally, when you want to automate the V&V process and run these test cases on bulk on other devices/software versions of your project, this toolkit will compare the results of the current run with the original, it will hint at the differences and will generate a report that will be used to validate your new version of software or your newly manufactured copy of your hardware.

Based on this, the program has three modes:

• SETUP mode. On setup mode you define all the test cases. You can order them, give them a short description, a category, a number of repetitions, you may enable or disable it.
• BUILD mode. On build mode you can run all test cases at once or one at time. For each one, you will define what conditions are to be satisfied to considerate the result as successful. In example: you may specify that the test is successful if and only if the output when run again is exactly the same; or, you can specify which parts are to be the same. Once done, you'll be able to generate a report of the expected behavior of your device.
• TEST mode. On test mode, all test cases are run and they get filtered by the rules defined on BUILD mode. After that, you may generate a report with the obtained results and it will point out the reasons that justify if each test case is successful or not.

Manual of this program

At the moment, you can check an example with photos and steps taken here!

External libraries

This project uses the following libraries:

• The UI is made with PyQt6, version 6.7.1.
• For the color scheme: qdarktheme, version 2.1.0.
• To export test cases to .xlsl files I use openpyxl, version 3.1.5.
• To bundle the program as an executable file: PyInstaller, version 6.10.0.

You can use this command to install everything:

$ python3 -m pip install PyQt6 pyqtdarktheme openpyxl pyinstaller

The project was built and tested on Python 3.10.8 64-bits.

To bundle as an executable

Run the following:

pyinstaller --onefile -w -i res/Logo.ico -n VVT src/Main.py --exclude PySide6

The last exclude is optional, it's in case you also have installed PySide6 on your environment.

Developing on Linux

To run the Python program in Linux (WSL), I had to install the following packages:

$ apt-get install ffmpeg libsm6 libxext6 libgl1-mesa-dev

This project is a work in progress.