Skip to content

Reduce CAM initialization memory footprint that is important for ne1024 simulation- #4102 - #1643

Open
sjsprecious wants to merge 2 commits into
ESCOMP:cam_developmentfrom
sjsprecious:fix_ne1024_init_oom
Open

Reduce CAM initialization memory footprint that is important for ne1024 simulation- #4102#1643
sjsprecious wants to merge 2 commits into
ESCOMP:cam_developmentfrom
sjsprecious:fix_ne1024_init_oom

Conversation

@sjsprecious

Copy link
Copy Markdown
Collaborator

This PR reduces the memory usage during CAM initialization by adding a new module named cam_shmem_mod.F90, which allocates the read-only looku tables once per shared-memory node and maps it into every rank on that node.

Then the following modules are updated to reduce duplicated fields/tables:

  • mo_jlongxsqy, rsf_tab, xs_o2b/o3a/o3b. Also changes the bulk broadcast from all-ranks to leaders-only, cutting the inter-node broadcast volume.
  • phys_prop — the modal, hygroscopic and volcanic-radius aerosol-optics tables.
  • soil_erod_mod — the source-grid erodibility field, now read on masterproc only.
  • tracer_data — one per-file buffer holding the global lat-lon source field in flight, so the node leader reads each field once instead of every rank reading it redundantly. Files already on the unstructured physics grid and zonal-average files are unaffected.

A few additional updates:

  • New finalization path (jlong_final, photo_final, physprop_final) so the windows are freed collectively from chem_final/phys_final.
  • cam_shmem_init() is called early in cam_init so the first MPI_Comm_split_type(SHARED) is paid at a controlled collective point rather than mid-physprop.
  • Initial-condition and topo files are closed at the end of cam_init, after their last use in phys_init, instead of being held open until cam_final.
  • tracer_data carries the overlapping time record forward in memory when the bracketing window advances by one record, instead of re-reading it.

This PR is done by Claude code under my supervisory.

Fixes #1640

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cacraigucar @jtruesdal this file is generic and not specific to CAM. My suggestion is to merge this PR with it for now (after addressing all the review comments). After that I will merge this code with a similar one in CTSM (ESCOMP/CTSM#4102), and bring it into CESM_share instead. Once we have a new CESM_share tag and bring it into CAM, we can then safely remove this code later.

@sjsprecious sjsprecious changed the title Reduce CAM initialization memory footprint Reduce CAM initialization memory footprint that is important for ne1024 simulation- #4102 Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants