Skip to content

fms2_io: Add collective writes#1733

Merged
vithikashah001 merged 14 commits into
NOAA-GFDL:mainfrom
J-Lentz:add_collective_writes
Aug 21, 2025
Merged

fms2_io: Add collective writes#1733
vithikashah001 merged 14 commits into
NOAA-GFDL:mainfrom
J-Lentz:add_collective_writes

Conversation

@J-Lentz

@J-Lentz J-Lentz commented Jul 14, 2025

Copy link
Copy Markdown
Contributor

Description
Adds parallel writes to fms2_io for 2D, 3D, 4D, and 5D cases with a new unit test.

Fixes #1715
How Has This Been Tested?
Builds with ifx 2025.2.0 on the AMD box and the test passes.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules
  • New check tests, if applicable, are included
  • make distcheck passes

Comment thread fms2_io/include/domain_write.inc
Comment thread fms2_io/include/domain_write.inc Outdated
@uramirez8707

Copy link
Copy Markdown
Contributor

As we discuss on Monday, if using collective mpi netcdf the code is going to put out a NOTE stating that the io_domain will be ignored and only write out 1 file.

These are some other changes that are needed:
uramirez8707@6999342 This is so that the dimensions are registered with the correct size (the compute domain rather the io compute domain)

uramirez8707@5908a72 This is so that the code doesn't write the domain decomposition attribute since it is not needed since there is no combining

@uramirez8707

Copy link
Copy Markdown
Contributor

Please include this commit so that there is a function that we can use to determine if a file is using collective mpi netcdf:
uramirez8707@c0f668b

I found this useful when implementing this in the diag manager

Comment thread fms2_io/include/domain_write.inc Outdated
buffer_includes_halos, msg="file:"//trim(fileobj%path)//" and variable:"//trim(variable_name))

c = 1
e = 1

@uramirez8707 uramirez8707 Aug 7, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than initializing e to 1, do

  e(1:2) = shape(vdata)

And for the 3D case,

e(1:2) = shape(vdata)
e(3) = 1 ! unlimited dimension

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a requirement for the last index to be the unlimited one? Will this have any effects on future efforts to implement generalized indices?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we've been assuming so far in the generalized indices work (at least, I did in the test that I worked on) that the unlimited dimension would always be last.

It's not clear to me if this is a hard requirement. It seems like it might not be with NetCDF 4, but the NetCDF documentation also says (https://docs.unidata.ucar.edu/netcdf-c/4.9.3/unlimited_dims.html):

For programmers, the unlimited dimension will correspond with the slowest-varying dimension. In C this is the first dimension of an array, in Fortran, the last.

Comment thread fms2_io/fms_netcdf_domain_io.F90 Outdated
Add parallel writes unit test and address Uriel's review comments.
@J-Lentz J-Lentz marked this pull request as ready for review August 14, 2025 20:49
Comment thread fms2_io/fms_netcdf_domain_io.F90
Comment thread test_fms/fms2_io/test_collective_io.sh Outdated
rem1776
rem1776 previously approved these changes Aug 18, 2025

@uramirez8707 uramirez8707 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very minor documentation changes, but looks good otherwise.

Nice work on the tests!

Comment thread fms2_io/fms_netcdf_domain_io.F90
Comment thread fms2_io/fms_netcdf_domain_io.F90
Comment thread fms2_io/include/domain_write.inc
@vithikashah001 vithikashah001 merged commit d8a19fd into NOAA-GFDL:main Aug 21, 2025
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FMS2io implement mpi collective netdft writes

5 participants