2023.06.15 Meeting Notes

Agenda

Individual/Group Updates

PG

Two meetings: split talk on physics + parthenon/athena-pk. Lots of interest in Parthenon. Documentation may be high priority based on this experience. Plan to clean up documentation, get full getting started guide/FAQ.
Will be around next two weeks.
Many people not thought about exascale computing. One you show people scaling plots, then they care.
Status of more coord systems?

BW

FG

Getting Athena-PK sims to run on frontier. Problems with ascent.
Tutorial + intro to Parthenon-hydro for LANL co-design summer school. School is Type 1a burn front. Parthenon hydro. Adding Riemann solvers, singularity-eos. Gravity solver.
Will try to get tutorials approved to share.
Shengtai's coords stuff is in AthenaPK. That requires copyright approval to move out. Bottom line is PG has collaborator to start coords from scratch. All people in T-2 want curvilinear coords. Could coordinate with external person + shengtai.

BP

Will look again at Prolong/Restrict ops. Plans to add Olivares+ prolong ops for face-fields.

LR

Enthusiastic thumbs up to Ben's proposal
Debugging some interaction between prolongation and sparse that causes small differences in behaviour between different numbers of ranks for sparse + AMR. No issues for dense vars. Weird interaction that happens after AMR in cases where you have comms over rank boundary and you get no data in ghosts. But when the two blocks are on the same rank, you get data in the ghosts. Doesn't understand yet why that's happening.
More with using all face/edge/nodal fields. Added vector potential-based MHD to parthenon hydro. Seems to work very nicely for uniform meshes. Haven't tried it with AMR. Pretty sure it'll work with AMR, because div(B) automatically maintained. Hoping to put up a PR finishes up the last things required to do face/edge/nodes with AMR.
LANL team had discussion about what sparse/all packs should look like. PR up where packs separated into pack ops and pack descriptors. More performant since descriptors can be more expensive. Packs then can be created more efficiently. Also means users can write functions that generate pack descriptors that behave in different ways. Trying to leverage that to get riot on parthenon develop.
Constant derivative boundary conditions and expose generic BCs. Makes it easier to expose BCs for different fields.
PGrete comments that perhaps missing fence in nested/hierarchical parallelism?
Same-to-same communication, in parthenon develop, dealloc count was not passed around.
Tests caught it. Tests are good.

PM

JMM

Debugging
Added more generic params read/write so you can restart from regression tests
Added read/write unit tests
Will re-bless gold files
Other people also interested since lots of people "abuse params"
FG suggests customizable IO for params
PG concerned compatibility/incompatibility between different packages. If mesh-level variables then we'd have to re-build packages. So supercharged params maybe is not clean but does do the job.
Forrest will look. If not done by Teusday to ping. Ben Prather will also look.

We often wanted to wait for releases but things always in flight
More tags is helpful
Do we want to change versioning from "numbered versions" to "year-month" just dump the code every two months
JMM: that seems simple which I like
Plan will be year-month-patch for spack
PG will look and then if it works out, just go for it.

Big question is: do people have any introductory material that they are happy/willing/allowed to share?
PG will take lead on fixing things
JMM large fraction of Athena++ docs that we can probably crib from with permission
BP has KHARMA documentation he's willing to share: https://github.com/AFD-Illinois/kharma/wiki/Parthenon#parthenon-datatypes

PG to help BW on user work before output PR
Athena PK people needed for sparse pack update (FG), constant derivative boundaries (PG).
BP and FG to look at Params