Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regroup tests in test suite #2441

Open
penelopeysm opened this issue Dec 18, 2024 · 2 comments
Open

Regroup tests in test suite #2441

penelopeysm opened this issue Dec 18, 2024 · 2 comments
Labels

Comments

@penelopeysm
Copy link
Member

penelopeysm commented Dec 18, 2024

Following on from #2433, some of the entries that were separately split off now run in quite short amounts of time. See e.g. CI runs on #2419

We should regroup them as necessary to make sure that we aren't spawning more runners than necessary (as that slows down other CI runs https://docs.github.com/en/actions/administering-github-actions/usage-limits-billing-and-administration#usage-limits)

@penelopeysm
Copy link
Member Author

penelopeysm commented Jan 21, 2025

@mhauru Right now we have 7 'test groups' which take a total of approximately 3.5 hours to run (https://github.com/TuringLang/Turing.jl/actions/runs/12879070698), so if we split them evenly we should be able to get CI to run in 30 mins (let's say 45 mins, because it depends on OS).

The main difficulty is that the Gibbs tests take approx 90 mins which is a bit of an outlier compared to the rest. Do you think we should we split up test/mcmc/gibbs.jl into smaller segments, or should we just bite the bullet and let CI run for 90 mins (in which case we could maybe consolidate some of the others and spawn fewer runners)? I'm kind of torn, but I think I'd lean towards the latter.

@mhauru
Copy link
Member

mhauru commented Jan 21, 2025

Also leaning towards the latter, because it's not obvious to me what would be a natural split of the gibbs.jl tests, except by AD backend, which I guess would eventually be taken care of by #2307.

Might it make sense to reorder the tests within gibbs.jl so that all the slow ones are at the end? This way you can, if you check the running CI job, more quickly see if one of the fast ones is failing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants