pest --parallel runs so much faster than phpunit (9 and 10) + paratest, and I don't know why

[mrozbarry]

preamble

My team and I were upgrading from Laravel 9 to 10, and thought that while we were doing the work, we might as well go from phpunit 9 to 10. Where we got stuck is phpstorm's test runner had a lot of problems tracking test progress and completions to the point it was unusable. Not a huge deal, running tests from cli worked fine, but we preferred to have the integration.

more on the question

So we installed the latest compatible version of pest, ran in parallel, and our 3k test suite (previously took 4-5min in phpunit+paratest) went down to 1.5 minutes. We didn't migrate any of our tests, they are still in the "phpunit" style, and the only real changes where the generated phpunit.xml file and using pest over phpunit.

We're also seeing ridiculous improvement from test coverage. phpunit+paratest+xdebug took a long time (maybe over 10 minutes), and pest does it in 2.5 minutes.

Now, I'm really really happy with the performance gain, but I'm concerned that it's running so fast because maybe it's somehow skipping tests. The successful test count is the same as displayed from phpunit, though. Maybe everything is fine.

I didn't really see any marketting around pest that boasts huge performance gains. The general laravel/php community seems on the fence with pest, and I haven't heard any performance gain success stories that quite have the same 50% time savings from 3k tests, and >100% time savings with coverage.