Skip to content

Allow reading fields with a constituent dimension#401

Merged
jimmielin merged 13 commits into
ESCOMP:developmentfrom
jimmielin:hplin/read_constituent_fields
Jul 16, 2025
Merged

Allow reading fields with a constituent dimension#401
jimmielin merged 13 commits into
ESCOMP:developmentfrom
jimmielin:hplin/read_constituent_fields

Conversation

@jimmielin

@jimmielin jimmielin commented Jun 26, 2025

Copy link
Copy Markdown
Collaborator

Tag name (required for release branches): sima0_06_001
Originator(s): @jimmielin

Description (include the issue title, and the keyword ['closes', 'fixes', 'resolves'] followed by the issue number):

Background

Certain fields written in the CAM snapshot (e.g., cam_in%cflx) have a constituent dimension. Reading such snapshot fields is necessary for testing vertical diffusion which needs this field.

Presently, the registry is configured to accept the definition of fields with a number_of_ccpp_constituents dimension but the initial value read infrastructure does not accept it with the error: cflx already has an initial_value, using it now. It also cannot be read from file: cflx has unsupported dimension, number_of_ccpp_constituents.

This PR enables the reading of 2-D variables with the dimension horizontal_dimension number_of_ccpp_constituents with the ability to trivially expand it to 3-D fields in the future.

Details

  • Because the CAM snapshot output constituent indices do not match the indices in the SIMA model, companion PR in CAM (cam6_4_101: Write cam_in%cflx for all constituents, not just Q CAM#1336) will output cam_in%cflx as cam_in_cflx_<constituent name> (i.e., cam_in_cflx_Q, cam_in_cflx_CLDICE, ...)
  • A new interface read_constituent_dimensioned_field will read fields with a constituent dimension, reading variables one-by-one with a name constructed from a "base name" (cam_in_cflx) + "constituent short name" (_Q, _CLDICE, ...)
  • Because CAM is not aware of standard names, constituent names are mapped from standard names to short names (Q, CLDICE, ...) through the registry.xml initial value definitions.
  • Unit tests have been updated

Tested by vertical diffusion work in progress code in SIMA.

Describe any changes made to build system:

  • write_init_files.py updated infrastructure to allow constituent dimension input

Describe any changes made to the namelist: N/A

List any changes to the defaults for the input datasets (e.g. boundary datasets):

List all files eliminated and why: N/A

List all files added and what they do:

A       test/unit/python/sample_files/phys_vars_init_check_constituent_dim.F90
A       test/unit/python/sample_files/physics_inputs_constituent_dim.F90
A       test/unit/python/sample_files/physics_types_simple_constituent_dim.F90
A       test/unit/python/sample_files/physics_types_simple_constituent_dim.meta
A       test/unit/python/sample_files/write_init_files/phys_vars_init_check_constituent_dim.F90
A       test/unit/python/sample_files/write_init_files/physics_inputs_constituent_dim.F90
A       test/unit/python/sample_files/write_init_files/simple_reg_constituent_dim.xml
A       test/unit/python/sample_files/write_init_files/temp_adjust_constituent_dim.F90
A       test/unit/python/sample_files/write_init_files/temp_adjust_constituent_dim.meta
  - new unit test (test_simple_constituent_dimensioned_var_write_init) for constituent-dimensioned input variable

A       test/unit/python/sample_files/physics_types_simple_initial_value.F90
A       test/unit/python/sample_files/physics_types_simple_initial_value.meta
  - fix missing files from previously introduced test

List all existing files that have been modified, and describe the changes:
(Helpful git command: git diff --name-status development...<your_branch_name>)


M       src/data/write_init_files.py
  - add checks for number_of_ccpp_constituents as alternate last dimension
  - if number_of_ccpp_constituents is a dimension, call read_constituent_dimensioned_field instead of read_field
  - rearrange location of const_props for use by above feature.

M       src/physics/utils/physics_data.F90
  - add interface read_constituent_dimensioned_field to read variables which have number_of_ccpp_constituents as dimension.

M       test/unit/python/test_write_init_files.py
  - new unit test (test_simple_constituent_dimensioned_var_write_init) for constituent-dimensioned input variable

M       test/unit/python/sample_files/write_init_files/physics_inputs_4D.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_bvd.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_cnst.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_ddt.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_ddt2.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_ddt_array.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_host_var.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_initial_value.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_mf.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_no_horiz.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_noreq.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_param.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_protect.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_scalar.F90
M       test/unit/python/sample_files/write_init_files/physics_inputs_simple.F90
  - updated sample files due to rearranging of const_props to top of subroutine.

M       test/unit/python/sample_files/write_init_files/simple_build_cache_template.xml
M       test/unit/python/sample_files/write_init_files/simple_reg.xml
  - fix typo and update hash for build cache template.

M       test/unit/python/sample_files/write_init_files/simple_reg_initial_value.xml
  - fix file name in xml.
  - fix allocatable should be allocatable (not pointer) if array has initial_value.

If there are new failures (compared to the test/existing-test-failures.txt file),
have them OK'd by the gatekeeper, note them here, and add them to the file.
If there are baseline differences, include the test and the reason for the
diff. What is the nature of the change? Roundoff?

derecho/intel/aux_sima: NLFAIL chemistry_nl - making baseline for #405

derecho/gnu/aux_sima: NLFAIL chemistry_nl - making baseline for #405

If this changes climate describe any run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced:

CAM-SIMA date used for the baseline comparison tests if different than latest:

@jimmielin jimmielin self-assigned this Jun 26, 2025
@jimmielin jimmielin added the enhancement New feature or request label Jun 26, 2025
@jimmielin
jimmielin marked this pull request as draft June 26, 2025 21:31
@jimmielin
jimmielin marked this pull request as ready for review June 26, 2025 23:04
@jimmielin
jimmielin requested review from nusbaume and peverwhee June 30, 2025 16:05

@peverwhee peverwhee left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks so much @jimmielin

A couple (hopefully) small changes!

Comment thread src/data/write_init_files.py Outdated
Comment thread src/data/write_init_files.py Outdated
Comment thread src/physics/utils/physics_data.F90
Comment thread src/physics/utils/physics_data.F90 Outdated
Comment thread src/physics/utils/physics_data.F90 Outdated
Comment thread test/unit/python/sample_files/write_init_files/physics_inputs_constituent_dim.F90 Outdated
Comment thread src/physics/utils/physics_data.F90
Comment thread test/unit/python/sample_files/write_init_files/simple_reg_constituent_dim.xml Outdated
@jimmielin
jimmielin requested a review from peverwhee July 2, 2025 15:31

@peverwhee peverwhee left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great!

@nusbaume nusbaume left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good @jimmielin! Just had a few questions and (hopefully) minor requests. Thanks!

Comment thread src/data/write_init_files.py Outdated
Comment thread src/data/write_init_files.py Outdated
Comment thread src/physics/utils/physics_data.F90 Outdated
Comment thread src/physics/utils/physics_data.F90 Outdated
Comment thread src/physics/utils/physics_data.F90 Outdated
Comment thread src/physics/utils/physics_data.F90
Comment thread test/unit/python/test_write_init_files.py Outdated
Comment thread test/unit/python/sample_files/write_init_files/physics_inputs_constituent_dim.F90 Outdated
@jimmielin
jimmielin requested a review from nusbaume July 16, 2025 18:30

@nusbaume nusbaume left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great now, thanks @jimmielin!

@jimmielin

jimmielin commented Jul 16, 2025

Copy link
Copy Markdown
Collaborator Author

Thanks @nusbaume and @peverwhee! Will merge this in once I run regression tests. Code is confirmed to work in my HB branch.

aux_sima_gnu_20250716140309: 14 tests
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_dme_adjust_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_dme_adjust_derecho NLCOMP
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_hack_shallow_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_hack_shallow_derecho NLCOMP
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_held_suarez_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_held_suarez_derecho NLCOMP
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_kessler_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_kessler_derecho NLCOMP
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_rk_stratiform_derecho NLCOMP
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_tj2016_after_coupler_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_tj2016_after_coupler_derecho NLCOMP
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_tj2016_before_coupler_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_tj2016_before_coupler_derecho NLCOMP
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_zm_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_zm_derecho NLCOMP
  SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_gnu.cam-outfrq_kessler_mpas_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_gnu.cam-outfrq_kessler_mpas_derecho NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_gnu.cam-outfrq_se_cslam_analy_ic (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_gnu.cam-outfrq_se_cslam_analy_ic NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM7.derecho_gnu.cam-outfrq_se_cslam_analy_ic (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM7.derecho_gnu.cam-outfrq_se_cslam_analy_ic NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FHS94.derecho_gnu.cam-outfrq_se_cslam (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FHS94.derecho_gnu.cam-outfrq_se_cslam NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_gnu.cam-outfrq_se_cslam (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_gnu.cam-outfrq_se_cslam NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FTJ16.derecho_gnu.cam-outfrq_se_cslam (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FTJ16.derecho_gnu.cam-outfrq_se_cslam NLCOMP

aux_sima_intel_20250716140307: 6 tests
  SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_intel.cam-outfrq_kessler_mpas_derecho (Overall: NLFAIL) details:
    FAIL SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_intel.cam-outfrq_kessler_mpas_derecho NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_intel.cam-outfrq_se_cslam_analy_ic (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM4.derecho_intel.cam-outfrq_se_cslam_analy_ic NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM7.derecho_intel.cam-outfrq_se_cslam_analy_ic (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM7.derecho_intel.cam-outfrq_se_cslam_analy_ic NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FHS94.derecho_intel.cam-outfrq_se_cslam (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FHS94.derecho_intel.cam-outfrq_se_cslam NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_intel.cam-outfrq_se_cslam (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_intel.cam-outfrq_se_cslam NLCOMP
  SMS_Ln9.ne3pg3_ne3pg3_mg37.FTJ16.derecho_intel.cam-outfrq_se_cslam (Overall: NLFAIL) details:
    FAIL SMS_Ln9.ne3pg3_ne3pg3_mg37.FTJ16.derecho_intel.cam-outfrq_se_cslam NLCOMP

@jimmielin
jimmielin merged commit 420b269 into ESCOMP:development Jul 16, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: Tag

Development

Successfully merging this pull request may close these issues.

4 participants