2024.07.25 Meeting Notes

Agenda

Luke:

PR up fixing regional tasking for iterative tasks.
Poisoning coarse buffer with NaNs on initialization has an impact on AMR solutions. Bad!!
We should add a regression test that communicates an exactly representable (by our prolongation and restriction operators) function on an AMR grid. Also would show an example of user defined boundary conditions.

Ben P:

Moved Kharma on to Parthenon develop, surprisingly easy.
Should make it easier to upstream a few things.
Starting to test multigrid there for divergence cleaning.
PG: Do you have performance numbers for GMG solver? BP: Old BiCGSTAB implementation took ~half an hour, MG precon-BiCGSTAB solver took about two minutes.

Patrick:

Artemis got bit by defining user boundary conditions that didn't work for coarse buffers.
Once that was fixed, they realized that sparse packs don't have the coarse coordinates even when PDOpt::coarse is set.

Adam:

Have Flash working with Parthenon, some stuff running through Parthenon on GPU and Flash kernels still running on CPU.
Going to write a generic dimensionality looping overload based on discussion in #1130.
PG: Some failed template resolutions related to loop patterns.

Philipp:

Still fighting io, but successfully wrote and restarted a 40 TB dump. Able to do in under two minutes, which is ok but could be better.
Tracking down an issue related to particles in outputs during restarts. As a result, implemented particles in PMD restarts to test if this is a problem with just the HDF5 implementation or something else.
PG: Has anyone seen similar issues with particles? PM: Have not seen this, but running at a different scale. Doesn't think he changed anything about HDF5 particle output, but he might have been the last one to touch this. JM: Also hasn't seen these issues with particles.
CI is now working on AMD GPUs, at least no issues over the last two weeks. Still some issue launching CUDA CI (but works by hand).
Doing MC inspired tracer particles, move from cell to cell stochastically based on fluid flows into and out of cell. May try swarm packs soon.

Jonah:

Has early user access to Venado, ran burgers benchmark there. H100s seem kind of nice, can build there, but can't run on more than one node yet. Code just built without any problems. PM: Got Parthenon running on multiple nodes last week at the hackethon.
Complaint: Matrix doesn't work for him (also doesn't work for Luke).
Jonah, Patrick, Chad, and Luke met with Petros Tzeferacos at LANL.
Asked Adam if Parthenon fortran bindings should be upstreamed. Interface maybe too specific to FLASH.

Hackathon:

PG: Have we signed up for another one? What months are available?
JM: One available in August and one in October. Everyone thinks October is better.

Non-WIP PRs:

PG: Next user call for OLCF will have discussion of ADIOS2 (backend for Open PMD), potentially interesting.