2024.12.12 Meeting Notes

Tentative agenda

• Individual/group updates
• Unified par_fors PR
• Review non-WIP PRs

Individual/group updates

BR

• chasing down to (cray) mpich errors (Invalid count error from MPI_Isend) which seem to be related to Kokkos (as the error goes away with 4.5)
• Shall we bump to Kokkos 4.5 <- yes
• Looking into adding FARGO scheme (to Artemis)
  • should this go into Parthenon?
  • might be of more general interest for some general infrastructure comoving grids

PM

• looking into units (and interaction between Parthenon/packages and downstream units)
  • whats' the best way to provide units
  • how to convert between
  • Idea: add to "ports of call" (as intermediate interface -- also used to abstract from Kokkos)
    • so that it can be used in other codes (like singularity-eos) without parthenon dependency
    • then units can be removed from Parthenon
    • General agreement this is the way to go

BP

• downstream dev and debugging ISend issue above

PG

• seeding/discovering performance differences (on Frontier) with different version of AthenaPK/Parthenon and Cray stack
  • also seen in KHARMA (with Parthenon dev) -- restarts help
  • so either Parthenon and/or Frontier/cray/mpich
  • given that it's better with the latest stack it may be related to communication
  • but issue did not show up on Lumi (uniform grid)
• seeing multilevel artifacts (in large, restarted SMR sims from a old bin with a new one)
  • don't show up in new, smaller runs
  • suggestion: start sim from scratch with new version at large scale (might this be related to comms)
  • suggestion: run at smaller scale with many blocks (trying to reproduce comm patterns/tags)
  • suggestion: create smaller reproducer with old code and restart with new

Unified par_fors PR

• performance hit of ~22% for hierarhical kernels
• should be investigated before merge