:sparkles: :hammer: Conclusive and Automated Testing

[WillXuCodes]

Overview

CI driven or at least script driven SITL/HITL test asset(s), allows for faster development and minimizes backtracking to fix bugs.

Some potential requirements for this:

• Hardware in the loop (HITL) asset like a V5 brain hooked up to something acting as a CI server to test something.
• A robust software in the loop (SITL) platform that can run directly on the CI server that kicks off a suite of tests just to sanity check that all the API still behaves like how we expect it to behave. Could probably use V5 Qemu that vexide developed for some of this.
• Every addition to the API requires test cases to this test suite.

Motivation

One massive issue with the mainline PROS development process is that there's not an emphasis on developing proper test assets anymore (and especially ones that aren't too manual or a pain to use).

With a longstanding project, developing automated and streamlined tests assets may seem like a pain but having HITL/SITL tests allow a team to measure twice, cut once and move a lot quicker than a team that doesn't.

Additional Information

There are unit tests written for each sensor/actuator in the PROS kernel repo somewhere, I'm sure these can be updated to fit Zest's API and then used in some form of automation.

This was something that was being worked on as a pretty big deal with PROS but I think it fell apart after I left :(.

It's admittedly not a "sexy" problem to work on but it's something that is crucial for project stability.

Simulation

We can use vexide/vex-v5-qemu, for proper simulation. Alternatively we can take the WPIlib route and compile for the host architecture.

Requirements

• ✨ CMake #10

[WillXuCodes]

http://github.com/ZestCommunity/ZestCode/tree/main/src/tests

Tests here, looks like they were copied over. Def not a complete suite though.

[tropicaaal]

We can use vexide/vex-v5-qemu, for proper simulation. Alternatively we can take the WPIlib route and compile for the host architecture.

To some extent. It's worth noting that the vast majority of the SDK has not been stubbed in that regard, and what is there is currently best-effort. Most of my focus is on other things, so beyond basic checks that the kernel boots at all, I probably wouldn't recommend this (yet).

[Exortions]

Wait, could we have a brain and hardware running somewhere 24/7 and a server to connect to so that anyone in the org can test software using real hardware at any time? (or am I completely misunderstanding what you are trying to say?)

[SizzinSeal]

we could and its not even that unrealistic, but we could just as well stub out the VEX SDK and run the tests natively, seeing as we wouldn't have any v5 device support without someone moving/removing actual hardware

[tropicaaal]

Wait, could we have a brain and hardware running somewhere 24/7 and a server to connect to so that anyone in the org can test software using real hardware at any time? (or am I completely misunderstanding what you are trying to say?)

Assuming a CI pipeline is setup, it should be priviliged access given that untrusted user code is capable of permanently damaging a V5 brain.

[ABUCKY0]

It's possible to require approvals for CI to run, so in theory you could use that to prevent user code from breaking a brain. T

[SizzinSeal]

i may or may not have a supply of like 30 brains that each have at least 18 busted ports. I don't think its even necessary lol

[ABUCKY0]

Bruh how?

[SizzinSeal]

i'm part of a really old robotics organization that's also the biggest organization in VEX robotics. We have lots of broken stuff

[ABUCKY0]

If you want to go really overkill, you could setup relays with the brain power cable and use an Arduino or Pi to disconnect the brain when the server sits idle for an extend period of time.

[ABUCKY0]

i'm part of a really old robotics organization that's also the biggest organization in VEX robotics. We have lots of broken stuff

oh yeah right